A composer has developed a method to transform sound through body movements.
Kenneth Stewart, a third-year Ph.D. candidate in music composition, has developed a way to manipulate sound files and create an improvisational environment by using a motion-sensing input device. Stewart uses the Kinect device, which was developed by Microsoft for the Xbox 360 game console, to create music while moving around a room. The device is used in conjunction with the library of sounds on Stewart’s computer and his self-developed software program called Max.
Using the Kinect camera, which has motion-sensing capabilities, the application locates 15 points on a person’s skeleton and places them on an infrared depth map on a computer screen. The application does not create musical notes, but instead uses a series of algorithms to activate the sounds. Loudness, pitch and intensity are all factors.
“The future implication [of this paradigm] is heading towards the visual,” Stewart said. “We’re basically looking at what happens when this paradigm is applied to visuals.”
Stewart, a classically trained cellist, said he was looking for a way to translate the same expressivity through technology as he does through his instrument.
“The expressivity is imprecise,” Stewart said. “Dealing with imprecision is part of what makes this paradigm expressive. That’s where music happens.”
Stewart and Thomas DeFrantz, professor of African and African-American studies and dance, collaborated on a project titled “Imnewhere,” which was shown in Duke’s November Dances last year.
In the piece, as DeFrantz dances and narrates, Stewart uses the application to score the dance, which consequently forms the basis for DeFrantz’s movements.
“I took an improvising environment, and we choreographed, literally, my motions to construct the piece,” Stewart said.
Stewart said that he and DeFrantz plan to expand their six-minute piece into a 20-minute performance.
The genesis of the application began with the notion that technology can be used to convey a certain idea, said John Supko, an assistant professor of music. He added that Stewart’s projects come from a desire to create new works of art and find new possibilities using given technology.
“I was very impressed by his work,” Vice Provost for the Arts Scott Lindroth said. “It brings excitement back to live performance.”
Lindroth said that the technology can open music to a new form of improvisation.
“We are able to make instruments nuanced for new technologies,” Lindroth said.
Supko believes there is potential in this realm of study, noting that there is a physicality of music that has yet to be explored.
“Because music is vibration, we’re going to be wearing our music,” Supko said.
Supko and Lindroth both believe that in the future, Stewart’s work will bring increased attention to the potential innovations at the intersection of music and technology. Stewart said he is not currently planning to sell or distribute the application.
Get The Chronicle straight to your inbox
Signup for our weekly newsletter. Cancel at any time.