Here is the new python (not super happy with the name, but it will do for now):
def hash_data(one,size):
logger.debug("hash-data one: " + str(one))
logger.debug("hash-data size: " + size)
try:
size = int(size)
except:
return ket("",0)
array = [0] * (16**size)
for x in one:
our_hash = hashlib.md5(x.label.encode('utf-8')).hexdigest()[-size:]
k = int(our_hash,16)
array[k] += 1 * x.value
logger.info("hash-data writing to tmp-sp.dat")
f = open('tmp-sp.dat','w')
for k in array:
f.write(str(k) + '\n')
f.close()
return ket("hash-data")
Now, I have an example in mind that would be good to visualize. Recall:
sa: load improved-imdb.sw sa: table[actor,coeff] common[actors] select[1,6] self-similar[actors] |movie: Star Trek: The Motion Picture (1979)> +-------------------+-------+ | actor | coeff | +-------------------+-------+ | James Doohan | 0.109 | | DeForest Kelley | 0.109 | | Walter (I) Koenig | 0.109 | | Leonard Nimoy | 0.109 | | William Shatner | 0.109 | | George Takei | 0.109 | | Nichelle Nichols | 0.109 | +-------------------+-------+Now, in the console:
sa: load improved-imdb.sw sa: |result> => self-similar[actors] |movie: Star Trek: The Motion Picture (1979)> sa: hash-data[4] |movie: Star Trek: The Motion Picture (1979)> sa: hash-data[4] "" |result> sa: hash-data[4] select[1,6] "" |result> sa: hash-data[4] common[actors] select[1,6] "" |result>Then we make use of this script($ ./make-image.sh tmp-sp.dat), and then we have:
BTW, I should mention. The spikes in the first three graphs correspond to movies, and the spikes in the last graph correspond to the original series Star Trek actors.
Update: one more step to another superposition:
-- find all the movies the 7 original series Star Trek actors starred in: sa: hash-data[4] movies common[actors] select[1,6] "" |result>
sa: how-many movies common[actors] select[1,6] "" |result> |number: 262>What were the top 30 of these?
sa: table[movie,coeff] 100 select[1,30] coeff-sort movies common[actors] select[1,6] "" |result> +---------------------------------------+--------+ | movie | coeff | +---------------------------------------+--------+ | Road Trek 2011 (2012) | 76.562 | | Star Trek Adventure (1991) | 76.562 | | The Search for Spock (1984) | 76.562 | | The Voyage Home (1986) | 76.562 | | The Final Frontier (1989) | 76.562 | | The Undiscovered Country (1991) | 76.562 | | The Motion Picture (1979) | 76.562 | | The Wrath of Khan (1982) | 76.562 | | Trekkies (1997) | 76.562 | | To Be Takei (2014) | 54.688 | | Generations (1994) | 32.812 | | Bug Buster (1998) | 21.875 | | Loaded Weapon 1 (1993) | 21.875 | | Backyard Blockbusters (2012) | 21.875 | | FedCon XXI (2012) | 21.875 | | The Captains (2011) | 21.875 | | Unbelievable!!!!! (2014) | 21.875 | | Coneheads (1993) | 21.875 | | The 6th People's Choice Awards (1980) | 21.875 | | 36 Hours (1965) | 10.938 | | Actors in War (2005) | 10.938 | | Amore! (1993) | 10.938 | | Bus Riley's Back in Town (1965) | 10.938 | | Double Trouble (1992/I) | 10.938 | | Jigsaw (1968) | 10.938 | | Man in the Wilderness (1971) | 10.938 | | New York Skyride (1994) | 10.938 | | One of Our Spies Is Missing (1966) | 10.938 | | Pretty Maids All in a Row (1971) | 10.938 | | River of Stone (1994) | 10.938 | +---------------------------------------+--------+And what does this look like?
sa: table[actor,coeff] select[1,9] 100 select[1,30] coeff-sort movies common[actors] select[1,6] "" |result> +---------------------------------+--------+ | actor | coeff | +---------------------------------+--------+ | Road Trek 2011 (2012) | 76.562 | | Star Trek Adventure (1991) | 76.562 | | The Search for Spock (1984) | 76.562 | | The Voyage Home (1986) | 76.562 | | The Final Frontier (1989) | 76.562 | | The Undiscovered Country (1991) | 76.562 | | The Motion Picture (1979) | 76.562 | | The Wrath of Khan (1982) | 76.562 | | Trekkies (1997) | 76.562 | +---------------------------------+--------+
sa: table[actor,coeff] coeff-sort actors select[1,9] 100 select[1,30] coeff-sort movies common[actors] select[1,6] "" |result> +--------------------------+---------+ | actor | coeff | +--------------------------+---------+ | James Doohan | 689.062 | | DeForest Kelley | 689.062 | | Walter (I) Koenig | 689.062 | | Leonard Nimoy | 689.062 | | William Shatner | 689.062 | | George Takei | 689.062 | | Nichelle Nichols | 689.062 | | Grace Lee Whitney | 382.812 | | Mark Lenard | 306.25 | | Teresa E. Victor | 229.688 | | Majel Barrett | 229.688 | | Catherine (I) Hicks | 153.125 | | Harve Bennett | 153.125 | | Merritt Butrick | 153.125 | | Gary Faga | 153.125 | | Stephen Liska | 153.125 | | Robin (I) Curtis | 153.125 | | Michael Berryman | 153.125 | | Brock Peters | 153.125 | | John Schuck | 153.125 | | Michael (I) Snyder | 153.125 | | Judy Levitt | 153.125 | | Todd (I) Bryant | 153.125 | | David (I) Warner | 153.125 | | Michael (I) Dorn | 153.125 | | Tom Morga | 153.125 | | Richard (III) Arnold | 153.125 | | James T. Kirk | 153.125 | | Christopher (I) Flynn | 76.562 | | Malcolm McDowell | 76.562 | | Patrick (I) Stewart | 76.562 | | Gene Roddenberry | 76.562 | | Phillip R. Allen | 76.562 | | Steve Blalock | 76.562 | | David Cadiente | 76.562 | | Charles (I) Correll | 76.562 | | Bob K. Cummings | 76.562 | | Joe W. Davis | 76.562 | | Miguel (I) Ferrer | 76.562 | | Conroy Gedeon | 76.562 | | Robert Hooks | 76.562 | | Al (II) Jones | 76.562 | | John Larroquette | 76.562 | | Christopher (I) Lloyd | 76.562 | | Stephen (I) Manley | 76.562 | | Eric Mansker | 76.562 | | Mario Marcelino | 76.562 | | Scott McGinnis | 76.562 | | Allan (I) Miller | 76.562 | | Phil (I) Morris | 76.562 | | Danny Nero | 76.562 | | Dennis (I) Ott | 76.562 | | Vadia Potenza | 76.562 | | Branscombe Richmond | 76.562 | | Doug Shanklin | 76.562 | | James Sikking | 76.562 | | Paul (II) Sorensen | 76.562 | | Carl Steven | 76.562 | | Frank Welker | 76.562 | | Philip Weyland | 76.562 | | Judith (I) Anderson | 76.562 | | Jessica Biscardi | 76.562 | | Katherine Blum | 76.562 | | Judi M. Durand | 76.562 | | Claudia Lowndes | 76.562 | | Jeanne Mori | 76.562 | | Nanci Rogers | 76.562 | | Kimberly L. Ryusaki | 76.562 | | Cathie Shirriff | 76.562 | | Rebecca Soladay | 76.562 | | Sharon Thomas Cain | 76.562 | | Joseph Adamson | 76.562 | | Vijay Amritraj | 76.562 | | Mike Brislane | 76.562 | | Scott DeVenney | 76.562 | | Tony (I) Edwards | 76.562 | | David Ellenstein | 76.562 | | Robert Ellenstein | 76.562 | | Thaddeus Golas | 76.562 | | Richard Harder | 76.562 | | Alex Henteloff | 76.562 | | Greg Karas | 76.562 | | Joe Knowland | 76.562 | | Joe (I) Lando | 76.562 | | Everett (I) Lee | 76.562 | | Jeff (I) Lester | 76.562 | | Jeffrey (I) Martin | 76.562 | | James Menges | 76.562 | | John (I) Miranda | 76.562 | | Tom Mustin | 76.562 | | Joseph Naradzay | 76.562 | | Marty Pistone | 76.562 | | Nick Ramus | 76.562 | | Phil Rubenstein | 76.562 | | Bob Sarlatte | 76.562 | | Raymond Singer | 76.562 | | Newell (II) Tarrant | 76.562 | | Kirk R. Thatcher | 76.562 | | Mike Timoney | 76.562 | | Donald W. Zautcke | 76.562 | | Monique DeSart | 76.562 | | Madge Sinclair | 76.562 | | Eve (I) Smith | 76.562 | | Viola Kates Stimpson | 76.562 | | Jane Wiedlin | 76.562 | | Jane (I) Wyatt | 76.562 | | Charles (I) Cooper | 76.562 | | Gene Cross | 76.562 | | Rex (I) Holman | 76.562 | | Laurence Luckinbill | 76.562 | | George (I) Murdock | 76.562 | | Bill (I) Quinn | 76.562 | | Carey Scott | 76.562 | | Jonathan (I) Simpson | 76.562 | | Mike (I) Smithson | 76.562 | | Steve Susskind | 76.562 | | Cynthia Blaise | 76.562 | | Cynthia Gouw | 76.562 | | Beverly Hart | 76.562 | | Melanie Shatner | 76.562 | | Spice Williams-Crosby | 76.562 | | Rene Auberjonois | 76.562 | | John (II) Beck | 76.562 | | John (III) Bloom | 76.562 | | Jim (I) Boeke | 76.562 | | Michael Bofshever | 76.562 | | Carlos Cestero | 76.562 | | Barron Christian | 76.562 | | Edward Clements | 76.562 | | BJ (I) Davis | 76.562 | | Douglas (I) Dunning | 76.562 | | Robert (I) Easton | 76.562 | | Doug Engalla | 76.562 | | Trent Christopher Ganino | 76.562 | | Darryl Henriques | 76.562 | | Matthias Hues | 76.562 | | Boris Lee Krutonog | 76.562 | | James Mapes | 76.562 | | Alan (II) Marcus | 76.562 | | David Orange | 76.562 | | Christopher (I) Plummer | 76.562 | | Brett (I) Porter | 76.562 | | Douglas (I) Price | 76.562 | | Jeremy (I) Roberts | 76.562 | | Paul Rossilli | 76.562 | | Leon Russom | 76.562 | | Clifford Shegog | 76.562 | | William Morgan Sheppard | 76.562 | | Christian Slater | 76.562 | | Kurtwood Smith | 76.562 | | Eric A. Stillwell | 76.562 | | Angelo Tiffe | 76.562 | | J.D. Walters | 76.562 | | Kim Cattrall | 76.562 | | Shakti Chen | 76.562 | | Rosanna DeSoto | 76.562 | | Iman (I) | 76.562 | | Katie (I) Johnston | 76.562 | | Jimmie Booth | 76.562 | | Ralph Brannen | 76.562 | | Roger Aaron Brown | 76.562 | | Ralph Byers | 76.562 | | Stephen (I) Collins | 76.562 | | Vern Dietsche | 76.562 | | Christopher Doohan | 76.562 | | Montgomery Doohan | 76.562 | | Dennis (I) Fischer | 76.562 | | Joshua Gallegos | 76.562 | | David Gautreaux | 76.562 | | David Gerrold | 76.562 | | John (I) Gowans | 76.562 | | William (I) Guest | 76.562 | | Leslie C. Howard | 76.562 | | Howard Itzkowitz | 76.562 | | Junero Jennings | 76.562 | | Jon Rashad Kamal | 76.562 | | Joel (I) Kramer | 76.562 | | Donald J. Long | 76.562 | | Bill (I) McIntosh | 76.562 | | Dave Moordigian | 76.562 | | Tony (I) Rocco | 76.562 | | Michael Rougas | 76.562 | | Joel Schultz | 76.562 | | Franklyn Seales | 76.562 | | Norman (I) Stuart | 76.562 | | Craig (VII) Thomas | 76.562 | | Billy Van Zandt | 76.562 | | Paul (III) Weber | 76.562 | | Scott (II) Whitney | 76.562 | | Michele Ameen Billy | 76.562 | | Celeste Cartier | 76.562 | | Lisa Chess | 76.562 | | Paula Crist | 76.562 | | Cassandra (I) Foster | 76.562 | | Edna Glover | 76.562 | | Sharon Hesky | 76.562 | | Sayra Hummel | 76.562 | | Persis Khambatta | 76.562 | | Marcy Lafferty | 76.562 | | Iva Lane | 76.562 | | Jeri McBride | 76.562 | | Barbara Minster | 76.562 | | Ve Neill | 76.562 | | Terrence (I) O'Connor | 76.562 | | Susan (I) O'Sullivan | 76.562 | | Louise Stange-Wahl | 76.562 | | Bjo Trimble | 76.562 | | Momo Yashima | 76.562 | | Steve (I) Bond | 76.562 | | Brett Baxter Clark | 76.562 | | Tim Culbertson | 76.562 | | Ike Eisenmann | 76.562 | | John (II) Gibson | 76.562 | | Nicholas Guest | 76.562 | | James Horner | 76.562 | | Paul (I) Kent | 76.562 | | Dennis Landry | 76.562 | | Cristian Letelier | 76.562 | | Joel Marstan | 76.562 | | Jeff (II) McBride | 76.562 | | Roger Menache | 76.562 | | Ricardo Montalban | 76.562 | | David Ruprecht | 76.562 | | Judson Scott | 76.562 | | Kevin Rodney Sullivan | 76.562 | | Russell Takaki | 76.562 | | Deney Terrio | 76.562 | | John (I) Vargas | 76.562 | | Paul (I) Winfield | 76.562 | | John (I) Winston | 76.562 | | Kirstie Alley | 76.562 | | Laura (I) Banks | 76.562 | | Bibi Besch | 76.562 | | Dianne (I) Harper | 76.562 | | Marcy Vosburgh | 76.562 | | Buzz Aldrin | 76.562 | | G.Z. Allen | 76.562 | | Robert (XII) Allen | 76.562 | | Thomas Anitzberger | 76.562 | | Michael Armendariz | 76.562 | | Thomas Bax | 76.562 | | Robert (I) Beltran | 76.562 | | Craig Berthiaume | 76.562 | | Jared Bird | 76.562 | | Robert Boudrow | 76.562 | | Denis (I) Bourguignon | 76.562 | | Richard (II) Bowen | 76.562 | | Brannon Braga | 76.562 | | LeVar Burton | 76.562 | | Miguel Carreon | 76.562 | | Richard Clabaugh | 76.562 | | Bruce (III) Clarke | 76.562 | | Thomas Clegg | 76.562 | | William Coble | 76.562 | | Rick Corley | 76.562 | | Justin Reid Cutietta | 76.562 | | Frank (I) D'Amico | 76.562 | | John de Lancie | 76.562 | | Brian Dellis | 76.562 | | Daren Dochterman | 76.562 | | Rey Duran | 76.562 | | Ron Duran | 76.562 | | Chris (I) Fleming | 76.562 | | Jonathan Frakes | 76.562 | | Daryl Frazetti | 76.562 | | Dennis Friday II | 76.562 | | Ross Gabrick | 76.562 | | L.D. Gardner | 76.562 | | Travis Gates | 76.562 | | Michael (III) Gay | 76.562 | | Adam Geiss | 76.562 | | David Greenstein | 76.562 | | Armando Paul Guillen | 76.562 | | Peter Haberkorn | 76.562 | | Dennis Hanon | 76.562 | | Steve (III) Hardy | 76.562 | | Scott (III) Harper | 76.562 | | Randall Hawthorne | 76.562 | | Steve (I) Head | 76.562 | | Edward Herndon | 76.562 | | Matthew Herra | 76.562 | | John Hurles | 76.562 | | Devin Irwin | 76.562 | | Edgar Jauregui | 76.562 | | Richard Koerner | 76.562 | | David (I) Koontz | 76.562 | | Stephen (I) Koontz | 76.562 | | Rich Kronfeld | 76.562 | | Gabriel Kerner | 76.562 | | Erik (I) Larson | 76.562 | | David (I) Livingston | 76.562 | | Gary (I) Lockwood | 76.562 | | Robert (IV) Lopez | 76.562 | | Stanley Lozowsky | 76.562 | | Adam Madden | 76.562 | | Logan Madden | 76.562 | | Geoffrey Mandel | 76.562 | | Douglas Marcks | 76.562 | | Jason (II) Mathews | 76.562 | | Robert Duncan McNeill | 76.562 | | Steve Menaugh | 76.562 | | Carl (I) Meyers | 76.562 | | Tim (I) Meyers | 76.562 | | Jason (I) Munoz | 76.562 | | Phil Murre | 76.562 | | Salvador Nogueda | 76.562 | | Robert (I) O'Reilly | 76.562 | | Marc Okrand | 76.562 | | Rick (I) Overton | 76.562 | | Harminder Pal | 76.562 | | John Paladin | 76.562 | | Ric Parish | 76.562 | | Mark Payton | 76.562 | | Brian (I) Phelps | 76.562 | | Ethan (I) Phillips | 76.562 | | Thomas (I) Phillips | 76.562 | | Adam (I) Philpott | 76.562 | | Robert Picardo | 76.562 | | Daniel (I) Pilkington | 76.562 | | James Pollnow | 76.562 | | Glen Proechel | 76.562 | | Michael Raffeo | 76.562 | | Russell (I) Ray | 76.562 | | Patrick Rimington | 76.562 | | Jon (I) Ross | 76.562 | | Paul Rudeen | 76.562 | | Tim (I) Russ | 76.562 | | Robert (X) Russell | 76.562 | | Timothy (IV) Scott | 76.562 | | Douglas Shannen | 76.562 | | Daniel (I) Shea | 76.562 | | David (IV) Silverman | 76.562 | | Jason Speltz | 76.562 | | Brent Spiner | 76.562 | | Tom (I) Stewart | 76.562 | | Rocky Stinitis | 76.562 | | Mark (II) Thompson | 76.562 | | Dennis Thuringer | 76.562 | | Barron Toler | 76.562 | | Kenneth Traft | 76.562 | | Fred Travalena | 76.562 | | J. Trusk | 76.562 | | Alois C. Tschamjsl | 76.562 | | Karl Van Der Wyk | 76.562 | | Matt Weinhold | 76.562 | | Jonathan (I) West | 76.562 | | Michael (I) Westmore | 76.562 | | Wil Wheaton | 76.562 | | Travis (I) Williams | 76.562 | | Wayne Wills | 76.562 | | Barbara (II) Adams | 76.562 | | Teresa Bailie | 76.562 | | Holly Barbour | 76.562 | | Morgan Barbour | 76.562 | | Roberta Barnhart | 76.562 | | Jennifer Bax | 76.562 | | Esther Becerra | 76.562 | | Viki Beyer | 76.562 | | Martha Bock | 76.562 | | Jolynn Brown | 76.562 | | Nicole Compton | 76.562 | | Denise (I) Crosby | 76.562 | | Melisa Dahl | 76.562 | | Melissa Dahl | 76.562 | | Roxann Dawson | 76.562 | | Evelyn De Biase | 76.562 | | Maria De Maci | 76.562 | | Evelyn Eastteam | 76.562 | | Ana Espinoza | 76.562 | | Terry (I) Farrell | 76.562 | | Lynn Fulstone | 76.562 | | Glenn Gadd | 76.562 | | Laurel Greenstein | 76.562 | | Shantell Hafner | 76.562 | | Debbie (I) Hanon | 76.562 | | Diana Harper | 76.562 | | Lisa (III) Harper | 76.562 | | Sharron Hawthorne | 76.562 | | Joyce Herndon | 76.562 | | Inge Heyer | 76.562 | | Penny Keane | 76.562 | | L. Grace Klitmoller | 76.562 | | Margaret Koontz | 76.562 | | Joan Letlow | 76.562 | | Jane Lostumbo | 76.562 | | Joyce (II) Mason | 76.562 | | Chase Masterson | 76.562 | | Marcella Mesnard | 76.562 | | Diane (III) Morgan | 76.562 | | Renee Morrison | 76.562 | | Kate Mulgrew | 76.562 | | Anne Kathleen Murphy | 76.562 | | Stephanie (I) Murphy | 76.562 | | Carroll Paige | 76.562 | | Cheryl Petersen | 76.562 | | Shelly Raffeo | 76.562 | | Sondra Reynolds | 76.562 | | Jessica Rimington | 76.562 | | Mary Rottler | 76.562 | | Hope Rudeen | 76.562 | | Tonya Saunders | 76.562 | | Lori Schwartz | 76.562 | | Lori Seol | 76.562 | | Susan (I) Shea | 76.562 | | Wendy (I) Shea | 76.562 | | Evan Shride | 76.562 | | Donelda Snyder | 76.562 | | Helen (I) Souza | 76.562 | | Linda Syck | 76.562 | | Deborah Taller | 76.562 | | Jeri Taylor | 76.562 | | Linda Thuringer | 76.562 | | Allison (I) Todd | 76.562 | | Deborah (II) Warner | 76.562 | | Pat Weisner | 76.562 | | Deborah Wheeler | 76.562 | | Cheryl (III) Wilson | 76.562 | +--------------------------+---------+And what does this look like?
sa: table[actor,coeff] drop-below[150] coeff-sort actors select[1,9] 100 select[1,30] coeff-sort movies common[actors] select[1,6] "" |result> +----------------------+---------+ | actor | coeff | +----------------------+---------+ | James Doohan | 689.062 | | DeForest Kelley | 689.062 | | Walter (I) Koenig | 689.062 | | Leonard Nimoy | 689.062 | | William Shatner | 689.062 | | George Takei | 689.062 | | Nichelle Nichols | 689.062 | | Grace Lee Whitney | 382.812 | | Mark Lenard | 306.25 | | Teresa E. Victor | 229.688 | | Majel Barrett | 229.688 | | Catherine (I) Hicks | 153.125 | | Harve Bennett | 153.125 | | Merritt Butrick | 153.125 | | Gary Faga | 153.125 | | Stephen Liska | 153.125 | | Robin (I) Curtis | 153.125 | | Michael Berryman | 153.125 | | Brock Peters | 153.125 | | John Schuck | 153.125 | | Michael (I) Snyder | 153.125 | | Judy Levitt | 153.125 | | Todd (I) Bryant | 153.125 | | David (I) Warner | 153.125 | | Michael (I) Dorn | 153.125 | | Tom Morga | 153.125 | | Richard (III) Arnold | 153.125 | | James T. Kirk | 153.125 | +----------------------+---------+And our final graph:
OK. I think it might be interesting to show them all at once, in sequence:
No comments:
Post a Comment