Christopher Rapahel’s life-changing moment happened at 1 a.m. on a mountain. At the University of California, Santa Cruz, where he was working on his B.S. degree, Raphael found himself up late one night working on one project or another. By the time he was ready to go home, it was late enough that all of the busses had stopped running. So, he began the long, steep walk from the top of what Brown calls a hill, but is really sort of a mountain, into town. After spending his life up to that point as a musician – an oboist who soloed with the San Francisco Symphony at the age of 17 after winning the San Francisco Symphony Youth Artist competition, a college student who had begun with a major in music, someone who had let music define him – Raphael knew it was time to change course.
“The most important thing that happened to me during college is I learned just a tiny little bit about computer science and mathematics and realized I didn’t want to be a musician anymore.”
Raphael promptly changed his major and graduated from UC Santa Cruz with a B.S. in computer science. Soon after, he attended Brown University, where he received a PhD in applied mathematics. Now, as a professor at Indiana University, Raphael is a pioneer in the informatics school. Still, despite the drastic change from musician to scientist/inventor, Raphael has spent his career working music and computer science into a compatible organism. Raphael (and Indiana University) calls it music informatics.
On August 4, 2008, Christopher’s proposal, “Real-Time Planning of a Conductable Orchestra,” was approved by the National Science Foundation, which granted him $450,000 over 3 years. Ephraim P. Glinert, the Program Director for the Human Centered Computing Program at the NSF, says that the agency aims to help fund projects that will advance fundamental and applied research – meaning they help people turn their inventions in to reality. The goal of this project, which Raphael hasn’t given a real name to yet, is to create software that, when fed live video of an orchestra’s conductor, processes the information into a musical performance. The program will create audio for the ensemble that is being led by the conductor presented through the video in real time.
“I think it creates a laboratory for experimenting with musical interpretation,” Raphael explained. “It gives immediate feedback on trying out a musical idea. There’s a huge advantage to communicating some ideas through motion rather than through sound and that motion allows you to anticipate ahead of time when something will happen where sound doesn’t do that nearly as effectively.”
When Raphael conjured up the idea for this software, the first step was finding a way to make it possible and, after receiving six previous grants from the NSF, the organization seemed to him like a logical place to seek funding. Raphael’s proposal was accepted by the NSF through a process of peer review that identifies projects of scientific merit.
Raphael isn’t sure how he’ll use all the money, but one purchase has already been made: a reproducing Bosendorfer piano, a brand that captures and collects the pianists’ precise use of the keys and pedals. Raphael explains that the piano, along with incorporating the quality of other well-known and expensive piano brands, is extremely compatible with different forms of technology and will allow for an enhanced real-time experience and a partner to the ensemble that the program will create.
The grant money will be distributed over three years, but that doesn’t mean the project will be complete in three years. Raphael believes his worst inventions and ideas resulted from a strict due date. Despite the absence of a firm timeline, there is a clear ultimate goal: to produce a computer system available for the general public to use. One of Raphael’s earlier projects, Music Plus One, has already reached a commercial stage. Music Plus One was a follow-up to Music Minus One, not created by Raphael, a ‘more familiar’ (according to Raphael) project, which makes a recording of a piece of music for soloist and accompaniment. The struggles associated with this piece of technology prompted him to develop MPO, solving problems such as inability to adjust to tempo changes and expressive gestures. MMO is all about the recorded accompaniment; MPO puts the spotlight on the soloist.
A few professors in the Indiana University Jacobs School of Music – Mimi Zweig, a professor of Violin and String Pedagogy, and Howard Klug, professor of Clarinet, to name a few – currently use MPO personally and with students. Zweig calls MPO a Symphonic Orchestra in a Box.
“You tell the program what you’re going to play and you start playing and it can hear you. It follows what you do,” Raphael said about MPO. “This is about the soloist experience,” Raphael said. He explains that the Jacobs School of Music and Indiana University will likely not be able to give most undergraduate a soloist opportunity. “So here we are, in this place where we have more qualified people per capita than probably any other town in the world and they mostly don’t get this experience.” In creating MPO, Raphael is not trying to replicate the ensemble experience. Instead, he aims to give the consumer the feeling of having a solo, “which is what most really yearn for because it’s a thrilling event when it happens.”
Raphael would know about the soloist experience, after all. Recalling his days as a primarily a musician, Raphael remembers thinking about music all day. He thought about when he was going to practice a piece of music, how he was going to get a reed to work, how to fix something on an instrument, how long it would take him to drive somewhere so he would be at a rehearsal on time. The transition into math and science had a profound effect on Raphael. “I thought about mathematics all day long from the moment I got up and I did it until I went to bed at night.” But this time, it was different. It was exciting, and it was what, in his heart, Raphael wanted to be thinking about.
Making progress toward becoming an inventor was a long process. Raphael had begun his post-secondary studies at Northwestern University studying music. After transferring to UC Santa Cruz and later changing his major to computer science, Raphael proceeded to Brown University, where he came of age as a scientist. Brown was the most formative influence on Raphael and he still has a tremendous allegiance to the school where he studied applied mathematics with professors such as Stewart Geman and Phil Davis, with whom he still keeps in touch and who are still an influence on his work. As a professor and inventor in music informatics, Raphael is now practicing applied mathematics at Indiana University.
“To me, what applied mathematics is essentially about is building this bridge between a deep wealth of mathematical knowledge and something that you could actually do with it,” Raphael said. In short, applied mathematics is inventing, which is what Raphael loves doing.