Efficient Mass-Scale Classifications

This is just the first post in what will be torrent of new algorithms I’ve developed for dealing efficiently with datasets comprised of large numbers of observations.

This set of algorithms implements the dictionary of states referenced in my original article on the topic (the latest article on the topic is here, which links back to the original), allowing for radically efficient comparison between very complex observations, in this case reducing a set of observations to a single dictionary of states.

The net effect of this particular set of algorithms (attached below) is to achieve compression, taking a set of complex observations, that could each consist of thousands of Euclidean datapoints (or significantly more), and quickly eliminate duplicate states, reducing the dataset to a set of unique observations.

The example in the code below takes a sequence of observations intended to model a simple example in thermodynamics, of an expanding gas, which is plotted below.

Expanding Gas

One sequence from the expanding gas dataset.

Each state of the gas consists of 10,000 points in Euclidean space, and there are 10 observations per sequence, with 10 sequences, for a total of 1,000,000 vectors.

Compression is in this case achieved in about 8 seconds, on an iMac.

This could be useful on its own, but I will follow up with another set of algorithms sometime this week that allow this initial step to facilitate fast prediction over these types of datasets, that consist of massive numbers of observations, even on consumer devices, which would otherwise be intractable, and then follow up with a comprehensive approach to natural language processing using these same algorithms.

The code necessary for this example is available here:

state_dictionary_insert

EMC_CMNDLINE

find_dictionary_index

All additional code can be found on my Researchgate page, under “Project Log”.

The Closing Sequence of Sketches of the Inchoate

The closing sequence of my book is admittedly weird, and I am plainly suggesting that I’m talking about the mythological figure Sigrid, and not the pop star, and this is deliberate. Though I don’t want to tell people how to think about the text, my read of the facts is that Ida is in fact Sigrid, and if you watch her video, “Strangers”, you’ll see a simply astonishing intersection between the text, and the video –

The plain implication is that Ida’s character is in fact a work of art, from my imagination, and the reveal is that Sigrid is in fact Ida, and that she is a real person. I’ll also note that the full name of the mythological Sigrid is, “Sigrid the Haughty”, which when read aloud, sounds like, “Sigrid the Hottie”, which is not only awesome, and true, but also a Marcel Duchamp reference, to L.H.O.O.Q, which is perfect structurally, since Sigrid appears after the end of the text, like a mustache on the Mona Lisa –

The surprise ending, after the work is already done.

Let’s begin with the opening line (00:18):

“It starts to rain, and we, we’re the broken beauties”, referencing the flood in the third chapter, as she rises from what looks like a canvas, covered in the shadows of a tree, wearing a yellow shirt, echoing the painting I made for Ida, which has a, “sculptural finish, with a thickness that is significantly raised off the surface of the canvas” (p. 69), as if Sigrid is a painting that’s come to life;

“Blindfolded, minds collide” (00:30), referencing the beers we buy in Sardegna, Ichnusa, the label for which features blindfolded silhouettes, echoing the trope that my true identity has been stolen from me, by bandits;

Birra_Ichnusa

Ichnusa Beer, featuring four blindfolded silhouettes on the label.

She mirrors the, “Michael Bolton” sequence (p.109), reading a paper, suddenly putting it down (2:24), staring, suggesting a secrete relationship between myself and Sigrid, and of course, she’s angry at the idea that I cheated on her with a Danish celebrity, Mø;

There’s a statue in the video that is plainly a male, stone version of, “Bacchante and Infant Faun”, (00:39) which is referenced in the New York sequence of the text (p. 46);

Bacchante_and_Infant_Faun_by_Frederick_William_MacMonnies_-_Hearst_Castle_-_DSC06204

Bacchante and Infant Faun, by Frederick William MacMonnies (1894).

The drapes hanging are colored like the white marble wall of the bar in Sardegna, which now appears as a soft fabric, that she simply walks through, suggesting visibility into the artistic process (00:48);

The stairs in the background reference the stairs in the studio in Copenhagen, again suggesting you’re now in the studio, getting a glimpse into how things work (01:13);

The giant rocks she’s dancing around (3:13) reference the, “giant sand, or salt crystals” from the artwork in the gallery sequence (p. 187);

Further, the canvas she’s standing in front of (3:37), lit up in green and blue lights, featuring trees, plainly also references the art work that she is likely to have made in the gallery sequence, though almost as if it’s another version, made to represent the beach at night, rather than during the day;

“Memories and photos, too easy to rewrite; left us lonely shadows” (1:27) –

These are simply not normal lyrics for a pop song, especially by someone this young, and it is plainly a subject dealt with explicitly in the text (p. 119).

Sigrid inexplicably, repeatedly screamed like a maniac at a concert, referencing the Eurovision sequence, where a man appears, “basically shouting into a microphone, in what strikes Jeff as most likely to be German”, (p. 117), though Sigrid is obviously a woman, screaming in English;

View this post on Instagram

xoxo screamo girl😚 thank you @sigridmemes

A post shared by Sigrid (@thisissigrid) on

Watching screens is a repeated theme in the text, as the output of cameras, as a play on observation itself, and the TV plays an inexplicably important role in the aesthetics of rooms, and in substance, providing information that is often critical to a scene, which is clearly referenced in, “Strangers”, and in fact, Sigrid holds a camera, as if watching someone, perhaps me (2:33);

The recording process itself, as a strange invention of modern life, that lets ghosts come back to life (p. 181), also plays a prominent role in the text, and in the video;

The water at the very opening of the video, which has no obvious connection to the rest of the video, references the water while we’re driving after the flood, which represents the power of external objects, like art, to trigger memories, “protecting our memory as well”(p.150);

The words just before each chorus are, “our story’s after the end”, and the final sequence is in fact after the end of the book (p. 214), which actually ends on page 212;

She smiles at the end of the video, alluding to something bordering on funny –

As if she knew that this would all be used in a spectacularly different context, that would inevitably blow someone’s mind (4:02);

Finally, there’s a guy holding a fan (1:05), with my haircut, again suggesting visibility into the artistic process that generated the text –

That you get so see how I did it.

She’s also wearing yellow, in front of a blue backdrop, which is the coloring from our kitchen tiles (p. 125), and the coloring of the sky during the flood (p. 145), and also the color of my scarf in the photo below, demonstrating my artsy do.

hair_art

Super strange art dude.

The closing sequence would therefore be Ida watching herself on TV, but to resolve this impossibility, given the fact that she, “doesn’t know the song” in question (p. 214), the TV would in this case be playing a song that Ida has not written in this life, or hasn’t written yet, alluding to my ideas on time, which suggest alternate possible realities, and communication between not only different realities, but also from the past into the future, and the future into the past, which is a thread running throughout the entire text, which is in fact itself told almost completely out of order.

This is plainly reflected by the fact that Sigrid has red hair, and green eyes, whereas Ida has blonde hair, and blue eyes, yet I’m saying they are in fact the same person, though there’s a moment in the video for, “Strangers”, where her eyes appear blue in color (2:57), echoing the narrative of the text itself, which is intended to be a sketch, and incomplete in terms of information, with glimpses into what could be interpreted as a memory, but perhaps a memory stored exogenously, by artifacts in the external world, that I’ve strung together, thereby creating an alternate reality, implied by our reality.

Sigrid also says, in the final song referenced in the text, “Dynamite”, “You see, my red is blue”, alluding to subjectivity in perception, despite my belief in an objective reality.

Moreover, the scene of the video for, “Dynamite” is plainly reminiscent of how I described the studio in our home in Copenhagen, with the only objects in sight being a piano and an amplifier (p. 122), in a setting that looks more like a home, than a recording studio. Further, she’s not really wearing much make up, referencing the scene in Carnegie Hall, and appears to be wearing roughly colorless lipgloss (p. 39). The scene also represents the culmination of a path towards freedom for both characters, since we’re both free economically, Ida is free of her trauma, and I’m free of my hatred, if only for the moment –

This places us in a moment that is beyond time and space, where strange things are possible, where location becomes permeable, which is echoed by the opening words to, “Dynamite”, which are, “Don’t know where I am with you; forgetting time and space with you.”

This is also echoed by the closing painting, “The Kiss”, by Klimt, which I’ve rebranded, as “Liebesträume, No. 541”, which in English means, “Love Dream, Number 541”, as if this entire story were just one instance of the relationship between Ida and I –

That we have different names, and different bodies in other outcomes, but nonetheless, the same soul.

The_Kiss

The Kiss, by Gustav Klimt (1908).

This is also echoed by the lyrics from Sigrid’s song, “Strangers”, where she says, “Think we got it, but we made up a dream”, and the use of, “Dreams”, by the Cranberries, during the sequence at the beach (p. 85), with the idea being, that if you love someone that much, then moments really do appear surreal, at times, as if you’re dreaming –

That the physical constraints of reality break down, leaving just the two of you, in your own space (p. 5).

This appears again in the sequence in Sardegna, where I look up to find, “a memory mounted into a ceiling” (p. 33), suggesting in the aggregate, that my relationship with Ida is allowing me to recover a, “stolen dream” (p. 33 and 215).

If you buy into the magic, then Sigrid and I somehow created this book together, despite being strangers –

That what I said about our relationship is physically true, in that Ida and I, “contribute to a moving portrait, that we share, together, as coauthors and spectators of an uncertain future, and a certain now.” (p. 105), perhaps so we could rediscover each other.

Finally, I’ll note that my plainly bespoke and artsy hairdo inexplicably appears at 1:05 in Sigrid’s video for, “Strangers”, the quote suggesting we wrote the work together appears on page 105 of my book, and “EDA” backwards is “ADE”, which when expressed in numerals is, “145”, and from a distance, it’s not unreasonable to confuse a, “4” with a, “0”, but like I say in the book, it seems as though we all have, “eye problems” (p. 198).

Sketches of the Inchoate – Playlist

I’ve put together an admittedly experimental form-of-art playlist to accompany my book, “Sketches of the Inchoate”, comprised of six songs, to echo the six chapters of the book.

I’m not going to provide any heavy analysis on the meaning, and instead, deliberately float it, allowing for ambiguity and interpretation, unlike the actual accompanying music, which has a fairly clear meaning in each scene, that I’ve generally worked through.

The overall sequence of this playlist is intended to operate as a metaphor for Ida’s character, transitioning from having a secret, confronting trauma, twice, and then ultimately transitioning to motherhood, and grace –

The moments when she stares out of windows, singing to herself, the idea being that the future communicates with the present, and so Ida can hear her future children singing to her, at times, pulling her towards the realization of motherhood.

The text of the book is available below –

Sketches of the Inchoate

Sketches of the Inchoate – Analysis of the role of the arts

The role of coincidence in the arts

I plainly make repeated use of the word “baby”, in songs, and in the text itself, without out any obvious explanation, or justification.

As an artistic device, the word, “baby” serves as a unifying trope across a huge range of genres, from the bronze sculpture at the Met, to Led Zepplin and Fetty Wap.

This is also clearly planted to contrast with the expectations of people in modern relationships, who are likely offended by the use of the word in these contexts, despite the frequent use of the word in contemporary music, including by a woman, Mø, who’s from a highly gender-conscious society, Denmark.

The implication being that superficially unrelated artists are perhaps a bit more honest about their feelings, and that this creates a unifying coincidence, forming a trope –

“Baby”.

This is also a play on associations, and what I think is a perfectly normal tendency to infantilize a partner, since it signals the ability to care for children, which is obviously a huge component of any long-term relationship.

As a practical matter, for the reader, it’s a signal that once observed gives an otherwise unrelated group of artists something in common –

“Baby”.

This is an exogenously realized variant of the same instant familiarity I allude to when I first meet Ida, which I took to an absurd, and realized level with the horse, deliberately creating a shared experience for both of us.

It is nonetheless as if we had something in common even before we met, though this is disrupted by external events –

Food guy and shoe guy.

Further, I realized after writing the, “London Grammar” sequence, that it’s plausible that Ida in fact made the work that I describe, “The Beach just Beyond the Woods”, though this was totally unintentional.

This kind of coincidence nonetheless suggests deliberate action, by someone, to the reader –

The work is obviously similar to the painting that I made for Ida, and the gallery as described sounds a lot like our apartment in Oslo, and I even focus on the door of the gallery, just like I did for our apartment.

The work arguably serves as a mood-changer for me in that sequence, solidifying my fidelity to Ida, since the memories it conjures color the rest of my night, though this is unstated, and implied.

This is an absurd variant on the idea that artists share ideas, without communicating –

In this case, I invented a character, Ida, who somehow, as a character in my head, arguably shared her work with me, and this was genuinely unintentional on my part.

Taking this idea of coincidence even further, I have a somewhat lazy and misshapen right eye, Biggie Smalls plainly has a lazy eye, Dale Chihuly has an eye patch, and Fetty Wap also has a damaged eye.

Noticing this, I intended this intersection in appearance to operate as a symbol for coincidence in the arts generally, except this time, as a symbol for the idea that artists can almost see each other’s works, and that we share ideas without knowing it, elevating the role of coincidence in the arts, as if chance altered our physical appearance, to let us know that perhaps we have something in common –

This is symbolized by the color blue, which serves the role of the light of mankind, and the light of ideas, with New York a powerful source of this light, symbolized by the Empire State Building.

And that this light is the invisible medium that facilitates this spooky, otherwise unexplained communication among artists.

This is plainly alluded to in the cover art, which is a work of street art, featuring an encircled eye, that I’ve noticed as a reoccurring theme in street art around lower Manhattan.

This is also plainly alluded to by, “Blue Train”, by, “So What?”, from, “Sort of Blue”, and Ida’s ability to suddenly see, “the music beneath the page.”, which is written in blue.

This is also consistent with, “The broken frame” sequence, presenting the body itself as art, that is both a conscious work by the person in question, but also subject to chance.

In this case, a group of otherwise unrelated artists including myself all have a similar physical feature, that is unusual, and I’m clearly putting myself in the role of the observer, who through mutual appreciation of a group of otherwise unrelated artists, creates the opportunity for a single narrative to reference all of our work, alluding to the intersection of our appearance, reducing both to a common physical feature, that operates as a symbol for coincidence in the arts.

In this case, a physical feature that we all have in common –

Eye problems.

The role of music

The fourth book repeatedly makes use of song lyrics that are explicitly incorporated into scenes, creating an artistic device that is tough to manage, since the lyrics are rarely a perfect fit for the scene, outside of the few moments referenced.

The overall idea is to take the use of music beyond how it’s used even in film, which is typically an emotional match for the scene, and instead elevate it to a technical match on the subject matter as well.

Ramble On

“Mine’s a tale that can’t be told,” alluding to some mystery as to my ultimate origin, with Ida noting the obvious similarity in appearance between my uncle and myself, and only naturally asking about the matter.

Then, “magic filled the air.”, providing her with an answer, as he performs his version of, “The Beast Sonata”, which is plainly alluded to by Ida hesitantly asking the question, twice.

He also has an absurd, black t-shirt, referencing me, and is smoking a cigar, referencing both Ida and I, and a shared experience we had at a cigar bar earlier in our relationship.

Ida is of course astonished, and this connects to the next scene, where we’re driving back, again alluding to poorly understood connections between artists, as Jimi Hendrix plays, and so the car is filled with shadows and ghosts –

Both the outlines and the souls of the past, that made the music we’re now listening to:

The past as immutable, and a light that shines upon those who listen closely.

Message in a bottle and Blind Faith

Though the lyrics aren’t explicitly referenced, the plain implication is that I’m sending out an SOS –

That art is in danger.

I think this is serious, and I think artists are also in personal danger, as the political enemies of reason and art use economics to stifle the arts, and perhaps worse.

The scene is meant to operate as a reclaim –

Where did the arts move?

Right here, in the middle of the street –

The kids show up, and own Bowery, again.

The What

I did both –

The dialogue of the text references the lyrics, and the lyrics foreshadow events in the text.

Christine references the lyrics, which I suggest was inappropriate, because she should know that I’m married –

I’m presumably wearing a ring.

Further, if you unpack what she said, it’s, “What do you ask the world for?”, and the corresponding lyrics are, “Fuck the world”.

As a result, her statement arguably changes the context of the song, suggesting quite plainly, in my character’s imagination, that she is saying that she wants to have sex with me, which I allude to in my response.

Further, the lyrics, “Like trees to branches”, my favorite line in the entire song, save for Method Man’s intro to the second verse, foreshadow the artwork to come, which is comprised at least in part of fallen wood and leaves.

The selection of, “The What” and, “Trap Queen”, was also deliberate, to follow the evolution of hip hop, from the sampled beats of the early days, to the highly produced, cinematic beats that we’re now accustomed to, that are often performed by live bands, arguably blurring the line between pop and hip hop, which I’m sure at least in part explains its commercial success.

Wicked Son

This is intended to operate twofold –

My character is obviously a bit wicked, and male, and so the song clearly references me.

There’s also a subtle reference to another Biggie song –

“He’s gonna be a bad boy.”

However, Christine is certainly wicked, arguably malevolent, totally indifferent to the fact that I’m married, and it therefore fits well with her character as well, in part in title, not in substance, since she is clearly a woman.

Deja Vu

This is wonderfully self-explanatory, and oddly enough, the image of the video on YouTube looks exactly what I imagined for Christine’s character –

A Swedish Ava Gardner.

Trap Queen

This works in title, not in substance, and actually sets the stage for a fantastic misunderstanding:

Christine thinks I acknowledge the song, because I like her, whereas I think she looks at me, because she’s admitting that she’s a trap –

We split the title of the song, I take, “Trap”, she takes, “Queen”, causing both of us to look at each other, creating perfect misunderstanding, given the same set of environmental facts.

This is something I allude to several times –

The Andreas fight sequence;

Thai Food;

The Mø misunderstanding.

The general point being that having the same information could still lead to errors, if you don’t share a context.

It also alludes to the power of art, which is to create a shared context for life itself.

This is why I wrote this story:

It’s for every American, precisely because the breadth of genres addressed is insane, and deliberately pedagogical in drafting, allowing anyone to simply pick up the book, and learn about everything, from information theory, acoustics, Zen Buddhist theories of psychological well-being, to Ravel and Fetty Wap, all in 200 pages, while unconsciously learning about Scandinavian culture, which is a part of the world I think Americans could learn from, despite the fact that they’re obviously basically all about as white as you can get –

They take care of each other, and don’t fight over nonsense, which is something that Ida and I are supposed to symbolize as a couple throughout the text.

Paper or plastic?

Who gives a shit –

We’re speeding home, because car, baby powder.

Everyone should know that they’re cared for, and that is the role of the state –

It is the last resort, to make sure society works properly.

Sketches of the Inchoate – Done, and Complete

I completed my story, “Sketches of the Inchoate”, which is now comprised of five books, all of which are available below:

Book 1: Sketches of the Inchoate

Book 2: Sketches of the Inchoate – Information and Belief

Book 3: Sketches of the Inchoate – Black Tree

Book 4: Sketches of the Inchoate – Analysis of Main Characters

Book 5: Sketches of the Inchoate – My Song for America

Book 6: Sketches of the Inchoate – My Song for America, Analysis