Hitting the Books: Teaching AI to sing slime mold serenades

Welcome to Hitting the Books. With less than one in five Americans reading just for fun these days, we've done the hard work for you by scouring the internet for the most interesting, thought provoking books on science and technology we can find and delivering an easily digestible nugget of their stories.

The Artist in the Machine: The World of AI-Powered Creativity
by Arthur I. Miller

Most of the time when we hear about AI, they're taking our jobs or putting us in jail or inflicting some other autonomic horror upon humans. But there's a second side to that AI coin. One in which machine learning algorithms show us skin suits the beauty of the natural form, even if it has been procedurally generated.

In this excerpt from The Artist in the Machine by Arthur I. Miller highlights the work of artist Eduardo Miranda. He's melded the minds of a slime mold and a CPU to create, well, music.

Eduardo Miranda and His Improvising Slime Mold

What will be the central processing unit of the future? —Eduardo Miranda

Eduardo Miranda wants to shake up musical composition. At the moment, he is interested in central processing units (CPUs). In today's computers, CPUs are silicon chips with circuitry that enables them to perform arithmetical, logical, and control operations. But supposing we go beyond silicon, beyond digital, beyond even a quantum computer? What about, for example, a bioprocessor that powers a biocomputer? Or a hybrid computer, powered by silicon plus a bioprocessor?

A bioprocessor processes biological material. Miranda's chosen bioprocessor is a slime mold called Physarum polycephalum, the "many-headed slime," a huge, yellow, single-cell organism packed with millions of nuclei. It is a mass of creeping, jelly-like protoplasm that is sensitive to light and spreads out over forest floors, eating fungal spores, bacteria, and microbes.

Physarum polycephalum has extraordinary electrical properties. Passing an electrical current through it makes it behave like an electronic component called a memristor. Memristors have a memory for current; their electrical resistance depends on the amount of current that has passed through them in the past. For Miranda, the most interesting thing about biological memristors is that they are not as precise as silicon-based digital ones. The mold's electrical output is related to the input but in ways that can be hard to predict. He considers it a "creative processor." Turn the pitch of a sound into electrical impulses and the slime mold will produce its own electrical response, which can then be turned back into music.

Miranda plays duets with the slime mold, which responds with enigmatic sounds produced on his piano. The piano is the interface between himself and his biocomputer. He entitled the first piece that emerged "Biocomputer Music: A Composition for Piano and Biocomputer." There is no electrical connection between the slime mold and the piano. Notes played on the piano are converted into voltage oscillations that are fed through the slime mold in its petri dish. The vibrations cause it to change shape, producing energy that is turned into electrical energy and sent to electromagnets above the piano's strings. The charged electromagnets attract the strings and make them hum, as if the mold is plucking them. Miranda has performed two pieces in collaboration with mold: "Biocomputer Music" in 2015 and "Biocomputer Rhythms" in 2016, in which the mold provided percussion.

Miranda grew up in Brazil and discovered computers as a teenager when his father brought home a Sinclair. He also studied the piano from the age of seven. He found it hard to juggle these two interests, AI and music. One hot and humid summer day, he visited the cool confines of the campus library and came across an article by Greek composer Iannis Xenakis, in which music was embedded in Venn diagrams, set theory, and logic, familiar to him from his studies in computer science. This, he realized, was the way to combine his interests in music and computers.

For his early research, he studied the way cellular automata behave in the Game of Life, a cellular automata system invented by the English mathematician John Horton Conway in 1970. In this game, a mathematical grid of cells follows simple rules about when a cell is "alive" or "dead." The result is a riot of patterns. Besides its mesmerizing powers, the Game of Life turned out to have multiple unexpected uses—as a tool for exploring the evolution of spiral galaxies; calculating pi; investigating how ordered systems emerge from complex systems; and looking into why, in a multiverse scenario, only certain universes are capable of supporting life. Conway had hit on something universal.

Miranda was enthralled by the patterns and how they evolved. "This is very musical," he thought. "It's how certain composers explore variations on a theme." He invented rules for how the patterns of a melody evolve over time, like the patterns in the Game of Life. "But," he thought, "instead of visualizing these patterns as mathematics, how about listening to them?" This he accomplished by looking for patterns in musical ideas: phrases, rhythms, and harmonic structure. The machine encodes them and turns the results into lead sheets or MIDI files. Miranda had to curate the raw musical ideas to render them more playable. The computer, he feels, can only be "an assistant to generate ideas."

Miranda notes that although neural networks can imitate musical styles, AI has not as yet produced any truly original compositions. This he feels is the next great step and perhaps can be accomplished with the help of his unpredictable slime mold collaborator. He is also interested in creating human-machine interfaces in which brain waves replace keyboards and voice commands to help the disabled express themselves musically.

In the introduction to his very personal book, Thinking Music: The Inner Workings of a Composer's Mind, Miranda contrasts composing music using AI systems (neural networks) with algorithmic approaches, generating music from mathematical descriptions of fractals or of chaos. He writes, "Aesthetically, the algorithmic approach tends to generate highly novel and unusual music, whereas the AI approach tends to generate imitations of certain known types of music. ... Both approaches have their own merits and pitfalls, but I tend to adopt the algorithmic one." The biocomputer is one such innovative algorithmic approach.

Looking back over the interaction between music and technology in the twentieth century, he muses, "Cage and Stockhausen explored technology which they didn't really understand. Now my generation understands technology and can use it to express our thoughts. It's experimental music but we now understand what the experiments are telling us. So let's make art now that is twenty-first- century music." Miranda adds that he is not interested in reading about how a walk on a sunny day inspired someone to write magnificent music. Rather, he writes, "what I want to know is how he composed it. How did such inspiration become music?"

Excerpted from The Artist in the Machine: The World of AI-Powered Creativity by Arthur I. Miller (The MIT Press, 2019).