Carlos Guedes, associate professor of music at NYU Abu Dhabi, has composed for orchestras and pop musicians, and just about everything in between. But recently, he has been working on developing computer programs that produce their own music.

Generative music programs are applications that create music according to pre-determined principles. They can receive an input from musicians — or even dancers — and respond to the input according to those principles. The degree of response is variable, so that programs can be either deterministic, and create music that is expected, or indeterministic, in which case it's harder to know what the program will produce. Guedes is most interested in indeterministic systems. "We give the rules to the computer," he explained, "and we have an idea what might happen, but we don't know for sure. And that's a good thing, because it makes the musician interact with the machine."

A system he created collects rhythmic information from the movement of dancers, called "musical cues," and translates this information into music. With body movements a dancer can control the tempo of a musical score, and a musician accompanying the score can respond to the changes; this way, "the musician can communicate with the dancer in real-time," Guedes said.

The process of creating a generative music application requires not only the art of musicians, but also the technological knowhow of computer scientists and engineers. Before coming to NYUAD, Guedes worked with a team of programmers at the Sound of Music Computing Group at the Institute for Engineering and Computer Science (INESC TEC) in Porto, Portugal. He described the team as a group of "hybrids," who, like him, are interested in working across disciplines. Guedes hopes to recreate a similar group here in Abu Dhabi: "The amount of creativity you can get from people with a bunch of different skills is really powerful," he said.

With the Portugal-based team he designed an interactive and generative music app for the Apple iOS operating system called Gimme Da Blues. Double bass and drums are played by the application, while users lead with trumpet and/or piano. Depending on how the user plays, the bass and drums respond with more or less syncopation and intensity. Users can change the music style, key, tempo, and the volume of the accompaniment. The app also lets users record and share their performances.

With Gimme Da Blues, Guedes and his team observed that people who didn't know jazz didn't play as well as people who did. He thinks that tools like Gimme Da Blues could be used to educate people in different music styles. "Especially here in Abu Dhabi where there are many different kinds of music, you could train students in Arabic, Indian, or African music," he said. This opens new doors for classroom learning: "Instead of having the traditional music appreciation class where students sit and listen to music, you could have them play along on an app, and they could create a gestural relationship with that music," he explained.

Another major potential application for generative music is the video game industry. Imagine you are a knight riding through a medieval forest. The game's music is light and mirrors the bucolic scene. Suddenly you are attacked by a band of thieves. The music changes, and mimics the pace of the ensuing struggle. "The game industry spends more money on music than Hollywood. But they are doing it in the traditional way right now, by using orchestras," Guedes said. (The game Spore, released in 2008, featured generative music by Brian Eno, but generative music in video games has yet to gain widespread acceptance in the industry.)

Now Guedes is thinking about other interdisciplinary projects that can be done here in Abu Dhabi. His latest idea is to film and record a dancer moving through the desert, and then remove the image of the dancer in post-production. The effect would be to see and listen to sand being transformed by a pair of invisible feet — only the result of the dancer's gestures would be seen and heard. Not so different from the way a generative music app could help guide the hand of a music student over virtual keys.