Category Archives: Research

Consciousness Emerges from the Archive: Dwarves, Twitter Bots, and an Infinite Library

dear reader

In 1941, Argentinian writer Jorge Louis Borges published a short story called The Library of Babel. The piece describes an infinite library full of chambers of uniform bookshelves with books that contain every possible combination of letters, and therefore every possible book. The Library of Babel is a work of literature that predates both the Internet and general purpose computing, yet it provides a potent way of thinking about how artists, poets, gamer designers, and citizens of a digitized world attempt to navigate incomprehensibly vast amounts of data. Faced with an effectively infinite amount of information, the distinction between discovery and creation becomes meaningless, and consciousness itself can be shown to be nothing more than an exceedingly rare—though not impossible—configuration of information.

The Library of Babel is a description of a setting more than a narrative. It describes an infinite fictional universe made up of interconnected hexagonal rooms. Doorways connect these chambers laterally, while ventilation shafts and small spiral staircases connect them vertically. The walls of the rooms contain uniform bookshelves, “each bookshelf holds thirty-two books identical in format; each book contains four hundred ten pages; each page, forty lines; each line, approximately eighty letters.” The inhabitants of the Library, known as Librarians, have deduced several important axioms about the books and the Library itself. The Library contains all possible combinations of letters, along with spaces, periods, and commas. The Library is “complete,” meaning that every possible combination of letters and punctuation appears somewhere in the Library once but is not repeated. The Library therefore contains all books that have been written and all books that could be written. It contains The Complete Works of Shakespeare, it contains a version of The Complete Works of Shakespeare where Romeo’s name is changed to Steve, and it contains The Complete Works of Shakespeare with one spelling error, and on and on. The Library contains the history of your life as well as innumerable false histories of your life. The Library contains this paper. The Library has no discernable organizing structure, however, so the vast majority of text appears to be complete nonsense, a jumble of letters. Many Librarians live their entire lives never finding even a shred of comprehensible text. The discovery of several coherent pages in a single volume is hailed as a monumental historic occasion.

When the Librarians first deduced that the Library contained all books, they were overjoyed. All the answers to all of their questions—the meaning of life, the origin of the Library, a catalogue of catalogues explaining the location of useful books—must all exist somewhere. Their exuberance was quickly replaced by depression when the denizens of the Library realized that while those meaningful and precious books did exist, they had no realistic hope of finding them amid the endless pages of nonsense. Some Librarians committed suicide; others destroyed scores of books in fits of rage.

Borges’ story is about the frustration and terror that comes from having access to boundless information with no way to parse it effectively. Having unorganized  access to everything isn’t helpful when we’re in search of something particular. In fact, everything is the enemy of the particular. The existential dread of The Library of Babel starts to look different, however, when confronted with the power of general purpose computing. Computers are machines that sort, organize, transfer, and store information. Computers can detect and predict meaningful combinations of bits of information. What happens when we have an infinite archive and the ability to search for particular things within it? What happens when we combine chunks of the archive ad infinitum? What happens when bits of the archive can sort and recombine themselves?

Tweets Without Authors

Screen Shot 2016-12-06 at 1.37.42 PMOne artist who is experimenting with what it means to sift through the scrum of endless information on the Internet is Darius Kazemi. A coder and technologist, Kazemi has created dozens of playful net art projects, many of them bot accounts on Twitter, entities that spit out bits of text generated by code rather than a human author. Inspired by Ian Bogost’s Alien Phenomenology, or What It’s Like to Be a Thing Kazemi realized that philosophers, rather than writing their ideas down, could build things that embody those ideas instead. He began making twitter bots that use code to scrape and recombine text. Metaphor-a-Minute takes words from an online dictionary and assembles them into the grammatical structure of metaphors. Most are difficult to make sense of, “a premonition is a warren: defenseless and tacit,” or “an impression is a mucus: nondomestic, rootlike.” While others seem to stumble on coherence, and maybe even self-reflexivity, “an algorithm is a neurosurgery: geothermal and wide-open.” Metaphor-a-Minute manages to create meaning out of entirely random words by forcing them into a format that prompts us to be generous in our search for coherence. Is an impression a mucus? No, not literally, but as a metaphor, it could work, depending how far we’re willing to go to imagine a connection.

Screen Shot 2016-12-06 at 1.40.17 PMOne of his most popular bots, Two Headlines, takes two unrelated current headlines from Google News and combines them into one statement while preserving their grammatical structure. Some recent examples: “How Amazon snatched huge customer Motorola away from California Mosques,” “COLUMN: Sign up for Skin Cancer while you can”, “MacBook Pro inmates rappel from jail in escape, two at large.” Similar to Metaphor-a-Minute, the bot sorts random content into a predictable structure: the news headline. It’s also very funny, relying on the humor trope of combining two incongruous yet topical elements.

Moving beyond tweets and into more lengthy territory, Kazemi started #NaNoGenMo, or National Novel Generating Month. It’s a spoof of #NaNoWriMo (National Novel Writing Month, nanowrimo.org), a social media hashtag and annual campaign to encourage people to try to write a novel throughout the month of November, even if they’re not experienced writers. Kazemi’s take on the project is to challenge other creative technologists to write code during the month of November that generates something that can technically be called a novel, using #NaNoWriMo’s definition of a novel as simply “50,000 words of fiction.” Kazemi set up a page on GitHub, a community website for sharing code, that explains rules stipulating that participants must share both their novel and their source code at the end of the month. “The ‘novel’ is defined however you want,” says Kazemi on the GitHub page, “It could be 50,000 repetitions of the word ‘meow’ (and yes it’s been done!). It could literally grab a random novel from Project Gutenberg. It doesn’t matter, as long as it’s 50k+ words.”

For the 2015 edition of #NaNoGenMo, GitHub user adregan created an 800,000 word novel called The Cover of The Sun Also Rises. He took a digital photo the cover of his copy of the Hemingway classic and converted each colored pixel into a verbal description, in order. It reads like this:

Brass. Brass. Brass. Brass. Brass. Brass. Brass. Brass. Brass. Drab. Drab. University of California Gold. Brass. Brass. Dark tan. Dark tan. Dark tan. Brass. Raw umber. Raw umber. Coffee. Dark brown. Olive Drab #7. Seal brown. Olive Drab #7. Olive Drab #7. Bistre. Bistre. Café noir. Raw umber. University of California Gold. Brass. University of California Gold. …

A more readable entry is user kevandotorg’s Around the World in X Wikipedia Articles. The code automatically generates a narrative that follows several characters traveling around the entire globe, starting and ending in London, all the while pointing out interesting historical factoids about each place scraped from geo-located Wikipedia articles. The final novel is 117,303 words long, and the characters visit 1,611 locations. A sample:

I remembered St Agnes Place. Unless I was mistaken, this was a squatted street in Kennington. Passepartout asked me if it was the first to be squatted, but I did not know. We agreed it was social/affordable housing.

“I imagine this is a place for housing.” said Passepartout.

 Near here was the site of Kennington Common. I could see that it was a large area of common land mainly within the London Borough of Lambeth. …

Kazemi’s Twitter bots and the #NaNoGenMo projects share something of the spirit of The Library of Babel, but updated for the internet age. These projects acknowledge that we’re sitting atop a vast mountain of text—much of it incomprehensible, provisional, mundane, unread, and unreadable—that just needs to be mined in interesting ways. If we think of Kazemi and his collaborators as Librarians in the infinite hexagonal chambers, they’ve overcome existential dread not by finding exactly what they’re looking for in the stacks of nonsense, but by inventing machines that selectively extract and recontextualize meaningful bits. What these machines find is not what their creators are looking for in a specific sense, rather they find—or perhaps create—something entirely new out of the archive’s detritus. The advantage of a networked archive, compared to a library, is that a network can be hacked.

The Sorry Story of @horse_ebooks

Kazemi is a well-known Twitter bot creator, but the most famous Twitter bot in the short history of the form was not his creation. @horse_ebooks began in 2009 as spam bot created by a Russian entrepreneur named Alexei Kouznetsov. The account, along with dozens of similar topic-specific spam-bots, was meant to drive traffic to an affiliate marketing website that sell ebooks. In order to avoid Twitter’s automated detection of spam-bots, the account tweeted links to the ebooks site only occasionally, and never interacted with other users. The majority of its tweets were poor algorithmic attempts to sounds like a human Twitter user, by clumsily scraping text from a cache of cheap ebooks and other sources around the web. The tweets were often sentence fragments, or were completely nonsensical, such as:

“Unfortunately, as you probably already know, people”

“TO CREATE THE SQUARE SHAPE Build the structure. Shape your watermelon. Shape the young watermelon.”

“SHORT OF BREATH? BREATH? BREATH? BREATH? BREATH? BREATH? COUGH COUGH COUGH COUGH COUGH COUGH TOO MUCH? SHORT OF BREATH? BREATH? BREATH?”

“As you might know, I am a full time Internet”

helicoptersThe account amassed legions of fans, eventually gaining over 200,000 followers. People obsessed over its enigmatic tweets. Fans created web comics, jewelry, and t-shirts that sold so well they couldn’t be kept in stock. Several reporters wrote lengthy reports about their efforts to track down the bot’s owner. Then on September 24, 2013, Susan Orlean wrote a short post on The New Yorker’s website announcing that @horse_ebooks was not a bot after all. She explained that the account was owned by Jacob Bakkila and Thomas Bender. The two friends also owned another mysterious source of internet content, the YouTube channel “Pronunciation Book,” which made hundreds of short videos showing the text of commonly mispronounced words along with audio of the correct pronunciation. Both projects culminated with a live performance in an art gallery that coincided with the launch of a third art project, an interactive video work called “Bear Stearns Bravo,” about the 2008 financial crisis. In the performance, Bakkila and Bender (along with Susan Orlean, oddly enough) sat in a gallery with three telephones, answering calls that came in to a number tweeted by @horse_ebooks that morning. They read a @horse_ebooks tweet to the caller, then hung up. The line received thousands of calls during the daylong performance, after which @horse_ebooks never tweeted again.

Orlean, writing an in-depth article about the project and its origins the following year, explained that @horse_ebooks had, in fact, started as a Russian spam-bot designed to sell low quality ebooks. Inspired by artists who used short bursts of text, such as Christopher Wool and Jenny Holzer, Jacob Bakkila wanted to perform as a spam-bot. “He wanted to attempt an identity inversion,” as Orlean describes it, “he would be a human trying to impersonate a machine that was trying to impersonate a human.” Rather than create a fake bot from scratch, he decided it would work best to take over an existing bot. He managed to track down Kouznetsov, @horse_ebooks’s creator, and agreed to buy $250 worth of ebooks in exchange for ownership of the account. Once he had the login, Bakkila updated @horse_ebooks manually, even at odd times, deciding to never automate its output. He kept tweeting links to the ebook sales site, which saw an increase in sales due to the bot’s growing popularity. Although he posted tweets manually, he never wrote a single one, they were all sampled from various places around the web. “There are so many weird, unindexed sites out there,” Bakkila told Orlean, “When you go down the rabbit hole of spam, it’s an infinity of infinity.” The profundity and humor of the tweets is often due to the fragmentary nature of copying and pasting bits of text, which is an effect Bakkila sought to preserve from @horse_ebooks’ spam-bot origins. He told Orlean, “I was trying to wrest wisdom from these wisdomless piles of information.”

Screen Shot 2016-11-27 at 3.30.18 PMEven very abstract @horse_ebooks tweets often have searchable sources. One tweet reads, “ORONGLY DGAGREE DISAGREE NO G G NO G G G G G G NO G G NEIEHER AGREE NOR DGAGREE O O O no O O no O O no O O no neither neither neither”. If you do a Google search of this exact text you get a Google Books result which is a fragment of a table showing a FedEx employee satisfaction survey, from a book called Make Their Day!: Employee Recognition that Works by Cindy Ventrice, published in 2003. The table is turned on its side, causing the optical character recognition of the scan to capture an odd series of characters and partial words. How Bakkila came across this slice of text is anyone’s guess.

Reactions to the revelation that @horse_ebooks’ tweets were part of an art project and not the accidental poetry of a machine were mixed. Writing in The Atlantic, Robinson Meyer called it “the most successful piece of cyber-fiction of all time.” Dan Sinker had a different take. Sinker is a writer and open internet advocate who anonymously ran the Twitter account @MayorEmanuel, a profanity-laden parody poking fun at Chicago Mayor Rahm Emanuel. Upon learning of Bakkila’s revelation, Sinker penned a blog post titled “Eulogy for a Horse,” where he lamented the loss of magic felt when this internet gem was revealed to be a long con. Sinker references the infinite monkey theorem, the idea that if a million monkeys were allowed to randomly bang on typewriters for infinity, they would eventually type the works of Shakespeare. The notion that an infinite, random recombination of letters will eventually produce texts that don’t seem random at all—that are in fact wise and poetic, even without an author—is at the heart of Borges’ The Library of Babel. @horse_ebooks was popular because it was funny and unpredictable, but it was magical because it appeared to be a real infinite monkey on a typewriter, arriving at beauty without intent.

The fact that @horse_ebooks had the identity of a beloved animal was part of what made it work, we sensed that it was an alien intelligence, foreign to us, but still wise and somehow empathetic. @horse_ebooks made people happy. Mourning its loss, Sinker writes:

I want to believe that beauty can be assembled from the randomness of life all around us.

I want to believe that a million monkeys can make something amazing.

God.

I really, really do want to believe.

But I don’t think I do.

And that feels even worse.

In Susan Orlean’s telling of what Jacob Bakkila was trying to do, we have to acknowledge that @horse_ebooks was an accomplishment of endurance and appropriation. But a well-executed net art project simply cannot compete with the original promise @horse_ebooks claimed: that the network had learned to write poetry about itself.

Uncreative Writing

There are many Librarians searching for a soul lurking in the archive. One prominent sifter of extant texts is the poet Kenneth Goldsmith. Goldsmith teaches a class called “Uncreative Writing” at the University of Pennsylvania where students are penalized for showing any kind of creativity. Instead, they are told to plagiarize, repurpose papers, steal identities, sample, plunder, and appropriate. He has found that the students are very good at this, they arrive as experts. For the final paper, they’re instructed to purchase a pre-written term paper, normally a violation of academic codes of conduct, and present it as their own. Rather than merely turning it in, students are required to give a talk on the paper to the class, adopting its arguments as their own, and defending it against critique from other students. The secret to the success of the class, according to Goldsmith, is that suppressing creativity is impossible. While undertaking seemingly mundane tasks such as copying pre-written text into their own documents, the students make creative choices about exactly what they select and why. The choice of what existing material to clone and what to ignore is very consequential, but it’s a creative act that’s often ignored. Goldsmith is placing his students in the same position as the Librarians in Borges’ story. Everything you could ever hope to read has already been written, the problem becomes how you go about finding it.

Goldsmith’s thinking is a response to the proliferation of the written word in the age of the Internet. The number of words we’re exposed to each day has exploded. Words are no longer confined to pages, they are plastic and fluid, they can be sorted, sifted, and poured into any container. Language even lies beneath the rich media that we think exists beyond text. Images, audio, and video are in reality just massive packages of code, a textual language that’s illegible and invisible to most of us, but is still loaded with syntax and meaning. “Writers,” according to Goldsmith, “are potentially poised to assume the tasks once thought to belong only to programmers, database minders, and librarians, thus blurring the distinction between archivists, writers, producers, and consumers.” Wandering the web is a very new experience in the scope of history, both aimless and purposeful. At one time, narrative held the promise that we’d be delivered through our reading to some satisfactory conclusion. But as we wander texts now, we’re caught in a blizzard of language that confuses, obfuscates, and overwhelms. In a short essay at the end of his book, Uncreative Writing, Goldsmith sounds like one of Borges’ despondent Librarians, declaring:

Narrative reflexes that have enabled us from the beginning of time to connect dots, fill in blanks, are now turning against us. We cannot stop noticing: no sequence too absurd, trivial, meaningless, insulting, we helplessly register, provide sense, squeeze meaning, and read intention out of the utterly senseless. The only legitimate discourse is loss; we used to renew what was depleted, now we try to resurrect what is gone. 

Digitizing the Infinite Archive

Borges’ story, in retrospect, looks like a thesis on the sampling, scraping, and “uncreative” writing practices of Kazemi, Bakkila, and Goldsmith, penned decades ahead of its time. Creation and discovery begin to blur. Literature, prophecy, and nonsense are all somewhere in the infinite archive, and finally the internet lets us invent tools with which to mine, process, and recombine the endless bits. Jonathan Basile, a PhD student in comparative literature, crossed the currents of contemporary e-literature and Borges when he created libraryofbabel.info, a digital version of the universe described in The Library of Babel. Basile devised a way to recreate a searchable, stable, algorithmically generated version of the infinite library. Users can open texts at random, select a particular book and page, or search for exact passages of text. Basile’s version contains 104,677 books, but it manages this without regenerating random books each time they’re accessed and without storing them all, which would be impossible. The website uses a pseudo-random number generator to power an algorithm that writes the text. In order to make the books searchable, Basile used an algorithm that’s invertable, meaning that it can both generate 104,677 unique books and find the exact location of any piece of particular text within those books, ensuring that the location of any particular bit of text will be the same every time.

Screen Shot 2016-12-06 at 6.44.31 PM

Searching for text on libraryofbabel.info can feel a little anti-climactic. No matter what you enter, a page is returned with the exact text swimming somewhere among a sea of nonsense characters. The search results page also shows the searched text on an otherwise blank page and in the middle of a page containing only real, albeit random, English words. These additional scenarios are exceedingly rare—you’ll never find a mostly blank page or a page full of readable English words when pulling up random pages—but they are no more rare than the page with the searched text in the middle of random letters, and they’re no more rare than any particular page of what appears to be complete nonsense. After all, a single page is but one out of 293,200 pages. In the “Theory” section of the website, Basile points out that one can only find text that has already been written. The tantalizing thing about the infinite library is the knowledge that amazing things that haven’t yet been written, or that have been written and are now lost, are in there somewhere. But there is effectively no way to find anything unless you know exactly what you’re looking for. “The most important experience the library can offer us,” says Basile, “is that of being overwhelmed by irrationality.”

Basile’s reading of The Library of Babel is that it’s a story about the erasure of the distinction between creation and discovery. Known books and unknown books that are comprehensible are in the Library, but jumbles of letters could hold significance that we don’t understand. Perhaps “yxve,chstqwggo ktqg,.xviw efpb.fa.” is not nonsense, but a potent message shrouded by a code, and the cypher for that code is found somewhere else in the Library. Language doesn’t lose meaning, instead it gains an infinite signifiability.

Since unwritten books are in the Library somewhere, did the Library “write” them? To this question, Basile writes,

I would emphatically distinguish libraryofbabel.info from any artificial intelligence, which is capable of recreating language in a context similar to its rational use. Babel is all expression in its most irrational, decontextualized form; I prefer to think of it as artificial unintelligence.

What’re Those Dwarves Up To?

Basile’s digital Library of Babel is an “artificial unintelligence” because it is static. It contains everything, but all its parts are locked into a rigid, nonsensical order. When near infinite archives can slice and recombine themselves, as computing allows, something resembling intelligence cannot only be found but can emerge. Since 2002, two brothers, Zach and Tarn Adams have been continually developing a very bizarre and complex computer game called Dwarf Fortress. The game is a little like Sim City and Warcraft real time strategy games, in that the player must design and manage a colony of dwarves in a procedurally generated environment. Rather than give the individual dwarves commands, the player queues tasks to be completed, and the dwarves eventually execute them, as long as they have sufficient food, alcohol, motivation, and materials. Players are responsible for setting up incredibly complicated webs of interdependence between the colony and the surrounding environment. To make gameplay even more challenging, Dwarf Fortress does not depict its world using graphics the way most modern computer games do. Instead, everything is rendered using colored ASCII characters on a black background. It looks like a DOS screen after a critical error, or like the streaming, seemingly nonsensical green characters on computer terminals in The Matrix films. Successful management of an unwieldy dwarf colony is ultimately impossible. Seasoned players accept that their fortresses will eventually succumb to starvation, goblin invasion, volcanic eruption, or any number of unforeseen disasters; and find that chronicling the travails of a doomed colony is one of the most rewarding aspects of the game.

800px-Sitefinder

Writing in Electronic Mediations: Comparative Textual Media: Transforming the Humanities in the Postprint Era, Patrick LeMieux and Stephanie Boluk discuss both the game’s approach to generative historical narratives as well as the fan culture that has developed around chronicling and sharing the tales of dwarf colonies. They call these narratives Dwarven Epitaphs. “Dwarven Epitaphs,” according to LeMieux and Boluk, “are monuments of human play and a powerful form of comparative textual media produced amid a computational wilderness.” The wilderness from which these narratives are drawn is the dense and unpredictable logic of events that unfold in the game. Zach and Tarn Adams have encoded every element of Dwarf Fortress—the dwarves, the animals, the weather, the history of the world—with complex narrative tropes and layers of logical causality. When all these semi-autonomous elements combine, they create a world that overflows with an epic, emergent narrative in which the player is invited to to play only a small role before being crushed by forces beyond her control.

When a player starts a new game in Dwarf Fortress, its algorithms generate an entirely new world from scratch. The player waits several minutes while eons of geologic and historic events are procedurally generated. LeMieux and Boluk note,

In one game set across a thousand-year time scale, Dwarf Fortress’s world generation will typically produce about fifty thousand noteworthy characters participating in over half a million events in thousands of locations. When confronted with these sprawling catalogs, stylistically reminiscent of only the barest of narrative forms—the calendar, the chronology, and the chronicle—the player is left to wonder how, for example, does a gathering cold front in the early autumn of the year 128 impact her activities over a thousand years later?

To account for these histories, each unique to a single save game file, would be impossible. The archives of these dwarven histories exist, but they’re incomprehensibly vast, leaving human players to marvel at each world’s complexity, serendipity, and blips of apparently emergent intelligence that bubble up.

On December 6, 2008, a user named Goldsie made a post on the official Dwarf Fortress community forums about the creation of a curious artifact in her game. She explained that a dwarf became “possessed” and began picking up huge amounts of stone and other materials scattered around the map. Thinking it was a bug, she ignored the erratic behavior, until after a year of in-game time had passed when a notification popped up that the dwarf had crafted an object of stunning complexity. “He turned thirty pages of stone, ores, shells and bars into one super statue,” wrote Goldsie, “when it was created the game froze for like 10 seconds just to spit out the history.” The dwarf named the object “Planepacked.” It was a dwarf-sized statue, made primarily of limestone but including dozens of other materials and featuring ornate carvings depicting most of the history of its procedurally generated world. The statue even contained 73 images of itself, meaning that its detail regressed into infinite fractals. When a dwarf crafts any type of object, the game produces a written description rather than an image—usually a concise way to describe a thing otherwise represented graphically only by a single ASCII character. The description of Planepacked, however, is 8,716 words long, or fourteen pages of single-spaced type. Here’s a sample from midway through the description:

On the item is an image of toads in Magnetite. On the item is an image of dwarves in Magnetite. The dwarves are laboring. The artwork relates to the foundation of Ravenlabors by The Sabres of Authoring of The Arrow of Tournaments in the early spring of 202. On the item is an image of Mafol Handletone the dwarf in Magnetite.

The question raised by Planepacked—this surreal and epic digital artwork—is simply this: Who made it? Zach and Tarn Adams created Dwarf Fortress, and in this sense they are deific clockmakers, winding up the world’s gears and stepping away. Goldsie discovered Planepacked, but she certainly didn’t create it. Planepacked could be considered a bug, which it technically was, as a more recent patch to the code prevents such creations from occurring again. On the other hand, the artifact is an example of exactly what Dwarf Fortress is designed to do: mash together so much complexity that something autonomous, unpredictable, and beautiful emerges.

As strange as it seems, it’s worth nothing that all 8,716 words of the description of Planepacked exist, in order, within Jonathan Basile’s libraryofbabel.info. In theory, the exact description is in Borges’ version, too. What was different about Planepacked in Goldsie’s Dwarf Fortress game is that it wasn’t sought out, it emerged. In a sense, Planepacked made itself, and then made itself known. Like a monkey on a typewriter completing his billionth page on his millionth year, something incredible coalesced from the swirling morass.

From Entropy to Consciousness

The infinite archive of The Library of Babel, along with experiments that blend randomness with endless archives of information—like those by Kazemi, Bakkila, Goldsmith, Basile, and the Adams brothers—all play with the concept of entropy. Entropy, a term borrowed from the second law of thermodynamics, is understood in information theory as the amount of disorder in a system. The entropy of a given message is equal to the average amount of information it contains. The second law of thermodynamics states that entropy can only increase, meaning that order gradually turns to chaos as natural forces seek equilibrium by achieving maximal disorder. The Library of Babel, as experienced by its forlorn inhabitants, is a stand-in for a complete entropic state. The information—the matter—of this universe is maximally disorganized, spread with perfect equilibrium across the infinite hexagonal chambers. And yet, there are the Librarians, agents who can read, collect, remember, and reorganize the tomes as they try to wrest order from the chaos. If the Library is the Internet, the artists, poets, literary theorists, and game designers we’ve looked at here are the Librarians, building tools, games, and robots to sort and reorder the Library’s depths. Unlike the Library, real information systems are not in a state of perfect entropic equilibrium, instead they’re chaotically recombining and realigning their parts at speeds we can hardly comprehend. The monkey on the typewriter is typing very, very fast. So it’s worth asking, when we glimpse things like Planepacked, whether something intelligent, or even conscious, could emerge from this swirl of connections.

monkeys-typingThe answer, according to a recent neurobiological study, is maybe. In a paper titled Statistical mechanics of consciousness: Maximization of information content of network is associated with conscious awareness, the authors describe a clinical study in which they observed the amount of entropy in among the neurons of patients who are in comas, sleeping, closing their eyes, and actively looking at things. They found that the highest state of consciousness, when subjects look at something to create a mental model of their environment, is associated with the highest level of entropy in the state of their neural network. It seems that a high level of chaos is needed for consciousness to function.

They go on to suggest that the emergence of consciousness itself may have a relatively simple physical explanation: our minds are the locations where matter has temporarily sorted itself into a sufficiently complex state on its way to maximally disorganized equilibrium. “The emergent features of cognitive phenomena that can be termed ‘conscious,’” say the study’s authors, “arise once an efficient web of connections endowed with certain complexity appears.” The key is not purely a maximum number of connections, resulting in everything connected equally to everything else, because that would be a static state, not unlike Borges’ Library. Consciousness might arise when the largest amount of possible connections are achieved within a given set of restraints. In the harsh language of thermodynamics, “consciousness (like biochemistry) may represent thus an optimal channel for accessing sources of (free) energy.”

As the Internet becomes something approaching an infinitely interconnected library, there may come a time when the bits align themselves in such a way that they emerge as something we recognize as truth, beauty, and being.

First Person History – Social Imaging in the Euromaidan Protests and Beyond

Molotov seflie

With the rapid rise in popularity of image-based social networks such as Instagram, the production and dissemination of political content on social media is becoming an increasingly visual activity. Throughout the Arab Spring and subsequent political upheavals, images captured and shared through cell phones played a key role in the way protest movements conceive of themselves and project that image to the world.

Alongside this phenomenon, social imaging more broadly, including the much-discussed “selfie,” has burgeoned into an enormous new realm of visual culture. People are creating and sharing images of themselves, their experiences, and their friends at a volume that dwarfs what was enabled by previous amateur photography technologies. Social imaging is often an effort to articulate, contemplate, and share the identity of the photographer. This activity takes on new dimensions when the producers of social images are taking part in historic and violent events.

In conflict photography there was once a clear division between subject, photographer and viewer. Now, all three roles blend into one. A single actor in a revolutionary event will act as photographer, subject, and primary viewer of an image. The audience for these images consists of a network of others wearing the same three hats. When the main function of social images is the formation and communication of individual identity, how does this function change when these images double as historical documents? How is history written differently when it’s done instantly and self-reflexively, by the same people driving the historical events?

Taking a selfie during a revolution is not just a way of showing the world what’s happening, it’s a way of creating an image used to contemplate that reality for oneself, which is then shared and archived, allowing that contemplation to continue through socially and through memory.

protester seflie

Euromaidan

On November 21, 2013, protesters took to streets in Kiev, Ukraine, angry that president Viktor Yanukovych abruptly decided against signing an Association Agreement and Free Trade Agreement with the European Union. The move would have signaled continued Westernization for the former Soviet Bloc state. Instead, Yanukovych opted to strengthen ties to the East, with Russian president Vladimir Putin. The protesters, upset by a stagnant economy and widespread corruption, called for the resignation of Yanukovych.

The protests grew substantially on November 30 as a response to police violence. Protesters and police each claimed areas of Maidan Nezalezhnosti (Independence Square), the main square of Kiev. The occasionally violent battles continued through the bitterly cold winter. In late February, violence escalated in the Square, Yanukovych and many of his supporters in parliament fled, allowing for opposition politicians to pass a series of reforms, effectively ending the standoff.

The protest came to be known as Euromaidan, a named coined on Twitter, which combines Euro- (for Europe) with Maidan (for the square). As with other recent protest movements, participants used social media to document and organize actions.

Around the time the Euromaiden protests were nearing their conclusion, I was experimenting with a web service called IFTTT.com, and acronym for If This Then That. It allows users to create “recipes” that allow different web services to interact with each other. A trigger in one service causes an action in another. It’s often used to create automatic back-ups, or small efficiency improvements. I noticed that the Instagram trigger allowed you to collect photos based on the location where the photo was taken. This allows you to essentially follow a location on Instagram, rather than a person. You select a location on a map, and anytime someone takes an Instagram photo there, it triggers the IFTTT recipe. I made a recipe to collect all the Instagram photos taken in Independence Square in Kiev and save them to a Google spreadsheet.

IFTTT

Over the last several days of the protests, I collected thousands of images. In general the images fall into three categories. First, amateur photojournalism, photos that exist primarily to document the scene. These include photos of burning barricades, first aid efforts, damage to buildings, makeshift memorials for the dead, etc. The second type is propaganda. These images are often not photographs, but are drawings or other graphics supporting the cause of the protestors. These are captured because they were uploaded from Independence Square, so they bear the GPS signature. The final type of images are the ones of primary interest to me, social images. I define social images as images that communicate the user’s identity, narrative, and style. These images include selfies, self-portraits taken at an arms length, but they also include portraits of friends, visual narrative, and other explorations of aesthetic identity. An important function of social images is their ability to help the photographer contemplate, archive, and re-contemplate themselves and their surroundings. This function is always present, but it becomes even more crucial when the circumstances surrounding the image become historic, violent, or otherwise hard to come to terms with.

Euromaidan seflie

The Selfie

To better understand social images, lets take a closer look at the ultimate social image, the selfie. In an essay titled, “Art at Arm’s Length: A History of the Selfie” art critic Jerry Saltz defines and unpacks the form:

A fast self-portrait, made with a smartphone’s camera and immediately distributed and inscribed into a network, is an instant visual communication of where we are, what we’re doing, who we think we are, and who we think is watching. Selfies have changed aspects of social interaction, body language, self-awareness, privacy, and humor, altering temporality, irony, and public behavior. It’s become a new visual genre—a type of self-portraiture formally distinct from all others in history.

Saltz points out that while selfies are casual, they are not accidental. Each image must be reviewed before it’s uploaded. This moment between capturing and sharing allows for “control as well as the presence of performing, self-criticality, and irony.” Selfies typically contain a knowing glance, a look of simulated eye contact with the viewer. Saltz compares this to “methexis” in ancient Greek theater, when the performer addressed the audience directly.

Some of the most noteworthy selfies are taken in unusual or inappropriate circumstances. Selfies taken at funerals by teens are common enough to warrant a blog, Funeral Selfies, that does nothing but track the questionable images. Many other examples of selfies in inappropriate situations exist, including touring a gas chamber at Auschwitz, in front of a car crash, and with someone about to jump off a bridge visible over the subject/photographer’s shoulder.

Funeral seflie

The most famous, or infamous, funeral selfie has never been seen. We only have third party documentation of the moment it was taken. President Obama, British Prime Minister David Cameron, and Danish Prime Minister Helle Thorning-Schmidt took a group selfie at the memorial service for Nelson Mandela. Saltz imagines the thoughts going through the heads of these world leaders as they snap this cheery and surreal photo. “It is totally incomprehensible, even to us, to be us … being here.”

These three world leaders, like the protesters in Independence Square, are writing both a personal and an historical narrative at once. In the moment, however, the weight of history only casts an uncanny shadow on the more immediate act of creating the image as a way of contemplating oneself in unbelievable circumstances.

I interviewed Alicia Eler, Selfie Columnist for Hyperallergic since June 2013. I asked her what she thought of my idea that selfies in war zones are first a tool of self-contemplation, and second a communication with the wider world. She said,

Selfies are recognizable as selfies, and those who take them in the middle of conflict zones do so as networked postcards. They at once appear as memories, documentary, and a quick hello, as if to say: ‘I was here. This is me.’ There’s a great selfie from the latest round of protests in Istanbul–protestors are thrown into the back of a van, and together they take a selfie that was instantly associated with Ellen’s academy awards corporate selfie because of the angle, the number of people in the photo, and the timing of both selfies. In this way, no matter where the selfie is taken–in the back of a protest van, at the academy awards, on a vacation, in a bathroom–it is more likely to be considered in relation to the composition of other selfies. And then the protest is against the selfie itself rather than what it captures.

If a selfie automatically situates itself in the wider context of other selfies, it’s worth expanding how this idea relates to social images more generally. Where else do social imaging and violent conflict intersect, and what can they tell us about the nature of each?

Rabih Mroué’s Pixelated Revolution

“The Syrian protestors are recording their own death.”

So begins a lecture/performance presented by Lebanese artist and actor Rabih Mroué presented in 2011 and 2012. Near the beginning of the Syrian civil war, Mroué found several clips on YouTube of protestors accidentally filming their own deaths. This was at a moment when the uprising in Syria was still viewed as the latest in a series of revolutions known as the Arab Spring. The violence captured in these videos was the very beginning of a long a brutal civil war that still rages today.

Rabih Mroué

The videos were taken because there was a sense that the crimes of the regime needed to be recorded, with the hope that president Bashar al-Assad would be held accountable. In one video, the cameraman stands on a balcony, franticly panning around in search of a soldier who is firing on nearby protesters. The video find the shooter, they share eye contact for a moment, then the image jerks wildly with a loud bang. Mroué calls the found videos “double shootings.”

One of the videos showcased and discussed by Mroué can be see here (warning, it is disturbing).

In the lecture, Mroué compares the aesthetic of double shooting videos to Dogme 95. Dogme 95 was an independent film movement that was active in Denmark in the mid 90’s. Filmmakers adhered to a strict set of guidelines, using very small budgets. Filmmakers eschewed special effects and other production techniques in favor of grit and veracity. The exchange of filming tactics among Syrian protesters operates as a parallel manifesto. In an interview, Mroué says,

[Dogme 95] stipulates that you should not record violent scenes, or weapons, because they don’t want to fake these things. So it’s not necessary to use them. For the Syrians, they add to this dictate insofar as the violent scenes being recorded are actually for real and the stipulation is also correct – do not record violence – insofar as the weapon could kill them and the scene of killing is thereafter real. There is no attempt to fake death here – it is all too real.

The Democratic Promise of Social Media

A persistent myth surrounding social media and political conflict since the beginning of the Arab Spring, or before, is that these technologies will have a democratizing effect. Once the world sees the actions of dictators through the eyes of the oppressed, the thinking goes, international condemnation and response will be swift and effective. This has not turned out to be true.

Social media is a propaganda tool that’s free for anyone to use. There’s no better reminder of this than the official Instagram account of Syrian president Bashar al-Assad. Widely considered to be a war criminal, Assad’s Instagram feed features political rallies, smiling diplomatic meetings in a lavish palace, and plenty of precious photos of the first lady performing humanitarian aid. Writing in the New Yorker, Emily Greenhouse considers the ethics of this banal and horrifying stream of images. She asks, “What does a social-media company do when a user known to be attacking civilians is blasting out feel-good content?” To find out, she asked Instagram. They refused to comment on specific users, but still managed to give a response with maddeningly circular logic. In general, Instagram will ban users who upload violent or hateful content. The determination is based largely on context, and “context” is limited to the content uploaded to the site. So as long as Assad’s Instagram feed presents the appearance of a benevolent and just leader—in other words, portrays effective propaganda—Instagram is just fine with it.

Social media as a democratizing force is also undercut by the way content can travel freely, even when it’s divorced from ideology. In the Washington Post, Neil Ketchley observes that revolutionaries share tactics through social media regardless of ideology. Recent Muslim Brotherhood protesters in Egypt have studied tactics of the Ukrainian protesters, despite the fact that their political affiliations couldn’t be more different. The Euromaidan protesters were fighting for increased Westernization and closer ties to Europe, while the Muslim Brotherhood is fighting for a return to power for Islamist leaders deposed by a more secular military. Ketchley quotes an anonymous Muslim Brotherhood protester, emphasis mine:

The Ukrainians know how to make big molotovs, so we watch how they use them and in which situations so that we can better defend ourselves against the police. We don’t care about the situation in Ukraine; we only hear a little about it on the news. But they know how to use molotovs to hold their square against the police.

Another revelatory example of social media use in war is the way the Israeli Defense Forces have recently used a number of platforms, including Instagram. A November 2012 assault on Gaza was documented in real time on multiple social media platforms. Updates came at each stage of the assault, announcing the killing of key enemy combatants, statistics on rocket strikes, and warnings to Palestinian civilians to avoid Hamas operatives for their own safety.

Huw Lemmey takes a critical look at IDF’s social media strategy, particularly Instagram, in his essay “Devastation in Meatspace.” This propaganda, like older examples, synthesizes a community that excludes others while protecting you, the intended viewer. Lemmey says the IDF employs a “visual regime based around firmly entrenched, conservative branding techniques.” Instagram has the dual effect of putting the war on a “consumer scale,” as well as positioning the conflict in a visual relationship with wars of the past through Instagram retro filters. The images don’t appear as raw digital evidence of the now. They are not the type of images that impress upon us the duty of deciding the moral culpability of those depicted. That work has already been done. The retro feel places the images safely in the historic narrative of the nation of Israel, alongside wars whose moral questions have already been settled. Lemmey characterizes the aesthetic of these images this way, “These are the photos you would take if you served in the IDF. We are just like you, and these military decisions are the ones you would take, if you were in our situation.”

IDF soldier on Instagram

Another tactic deployed by the IDF on Instagram is posting photos taken by individual soldiers going about their daily lives. They look sexy, fun, and hip. They suggest to the viewer, we can hang out, we can be friends, even if we don’t know each other, we should. The personal and political completely collapse into one. We’re no longer able to distinguish our feelings about what these soldiers are doing on a moral and geopolitical scale with who we think they are, and how fun it might be to hand out with them. Images of Palestinians, meanwhile, get further from us, less relatable, more other .

The Third Meaning

What distinguishes the IDF’s Instagram feed from my collection of all the Instagram photos taken in Independence Square during the final days of Euromaidan is the intentionality with which the images are created and shared. IDF’s Instagram is clearly propaganda. Some of the photos from Euromaidan are as well, but with others it’s hard to know. Who took these photos? Were they uploaded right away, or later? Who was the intended audience? If they conform to our idea of the recent narrative of Ukraine, who ensures that they tell the correct story, the photographer or the viewer?

Collecting and decontextualizing images in this way has a strange dual effect. On one hand, the conflict becomes very real, close, and almost personal. The rage and fear of the protesters becomes as genuine and immediate as everything else I see on Instragram, like my friends’ lunches and trips to the beach. On the other hand, this conflict is very far away and foreign. The images often seem raw and unfiltered, but also somehow conspicuously planned. They carry extra, incidental meanings. While employing smartphones to instantly capture, contemplate and share images of themselves in unbelievable circumstances, these photographers embed layers in these images beyond their intent and comprehension.

In Image Music Text, Roland Barthes identifies three ways images communicate meaning. The first is informational, the objects and people depicted. The second is symbolic. This is what the informational elements communicate through a shared understanding of the meaning of commonly interpreted signs. The third meaning is the obtuse meaning. This is similar to the symbolic meaning, but is open, incomplete, or perhaps broken. There is a sign, but we cannot give the signified a name. We can’t even be sure that it’s intentional. It’s the aspects of an image that point to meanings beyond language. Barthes talks about it this way,

The third meaning also seems to me greater than the pure, upright, secant, legal perpendicular of the narrative, it seems to open the field of meaning totally, that is infinitely. I even accept for the obtuse meaning the word’s pejorative connotation: the obtuse meaning appears to extend outside culture, knowledge, information; analytically, it has something derisory about it: opening out into the infinity of language…

Social Images from violent conflicts are most revealing in the aspects that are beyond their intended meaning. As they blend self-referential and documentary purposes, they gain an obtuse meaning that escapes the intention of the maker. This is because the intended audience of the images is the photographer herself and her social network, not history. They gain the third meaning when they become documents of greater historical and political significance. Signifiers that once pointed to some shared meaning are severed, giving the images an aura that comes from dislocation.