Peter Neubäcker, Hildegard Sourgens and Carsten Gehle paid a visit to our office to offer us a preview upcoming innovations in Melodyne. Not only did we get a glimpse of the cool things that the software will do, but we also got a chance to rub shoulders with true pioneers in digital audio technology - to speak nothing of this trio being three of the nicest folks we have ever encountered in the business.

Melodyne is the flagship software of the German software company, Celemony. Peter, Carsten, and Hildegard are the driving force of Celemony and the development of Melodyne. Peter is the concept thinker, Carsten is a skilled programmer and Hildegard completes the team with her business savvy.

While they are working on the next version release of Melodyne, they have a few other fascinating advances on the go as well. Peter has developing a cutting edge, guitar recording system where each of the 6 strings of an electric guitar can be routed to individual audio channels in your digital recording software. Incorporating the Melodyne pitch-shifting algorithms, this allows for individual tuning of a single guitar string even after the recording is already down on the hard drive! This will revolutionize the sound of the guitar, and the way guitar is recorded and produced. We heard it first-hand, and it was incredible.

I was really delighted to talk to the developers of Melodyne. Being an avid user of the product, I have already developed an infinite respect for the people who created the best audio manipulation software ever made.

Shawn Dalton: I know very little about computer programming, but I can see Melodyne far surpasses any of the other pitch-shifting software or hardware out there. Was it a supreme challenge to develop this software?

Peter Neubäcker: The challenge was in the new way of thinking which needed to be developed before Melodyne could exist. I didn’t start out as a computer programmer, and I don’t possess any real, gifted mathematical abilities. I figured things out just by doing a lot of experimentation, but changing my way of thinking was the real breakthrough in developing Melodyne.

I went right back to basic ideas and the natural processes which are related to the music we hear. I aspired to answer basic questions such as, what is sound? We always take sound for a given, but what is it really? What makes a random sound different from that which we call music? One question I kept asking myself over and over was - What is the shape of sound? There is a German word gestalt, which has a similar meaning to the expression “the whole is greater than the sum of its parts.” Gestalt has implications beyond shape and form.

Where I’m going with this is that the development of Melodyne was very much philosophically driven, and understanding the gestalt of sound was our task. One of the key ideas we had in the development of the program was that we had to attempt to consider the nature of sound outside of the measurement of time. We had to conceive of sound as happening in a domain where there was no such thing as time. Once we figured this out, the rest of the ideas fell into place.

This is the heart of the Melodyne programming philosophy. What we ended up with was a theory of sound where duration was not an element. My task was to try to define some laws of sound, formalize them, and then transcribe these formalized laws into algorithms. In practice we ended up with a program which could detect stability within a sound and any user defined parameters would apply only to the parts of a sound which were stable. When a given moment of sound can be described as stable, we think this moment of sound can also be described as possessing the quality of timelessness. When the dimension of time is removed form sound, then manipulating various qualities of that sound become easier. What we ended up with were algorithms which possess the characteristic smoothness of Melodyne.

Shawn Dalton: While in the process of conceiving of these algorithms was there a single moment of epiphany where things came to you, or is the the idea of Melodyne something that has always been with you?

Peter Neubäcker: Well, there was indeed a moment of epiphany. I was sitting together with a friend with whom I met regularly to discuss sound and ideas and we were pondering the question “What would be the sound of a stone?” We thought about the stone. A stone is an object which exists without time. As far as our human perception is concerned, time doesn’t matter to the stone. During a human lifetime a stone may exist and we may perceive no noticeable changes to that stone. Time is not a dimension that is important for the stone, nevertheless the stone has a shape and we say the stone has a gestalt. The question became something like...”Is there a way to extract the sound out of the stone’s gestalt?” And then, “What would be the sound of sound if there were no time?” This question then gave birth to a conceptual understanding which paved the way for the development of Melodyne.

Shawn Dalton: From this moment of epiphany, how long did it take to develop the Melodyne application?

Peter Neubäcker: It took about 3 years and a lifetime [Peter smiles]. It was a complete obsession of mine, and Carsten’s as well. Once the work of transcribing our ideas into algorthims was accomplished there was still a lot of work to follow. My original intent was not to develop something such as what Melodyne currently is. The first working piece of software was a type of graphic application where the user could paint sounds. After this it took about a year and a half to develop the first version of Melodyne which was released near the end of 1999.

Shawn Dalton: I’m curious about the way Melodyne has been packaged. It comes wrapped up in a very beautiful wooden box. My first reaction was that this seemed an extravagant and costly way to package software. Why the wooden box?

Peter Neubäcker: Well, we hired a third party company to design our logo and the packaging. When designing the face of Melodyne this company asked me about my personal history and I revealed to them that I considered myself primarily as an instrument builder. I’ve been an instrument builder for many years. I enjoy making guitars and other wooden stringed instruments and in fact my living room looks much more like a wood shop than it looks like the workplace of a software developer! In fact, I met my wife Hildegard because I used to host workshops for instrument building. She was a student in one of my workshops, and she has also contributed to Melodyne’s development. With this information our designer came up with the idea of the wooden box. We liked aesthetics and the suggestion that Melodyne has a strong connection with the tradition of instrument building. Melodyne is in fact a type of musical instrument, so we thought it quite appropriate.

Shawn Dalton
Club Cubase Magazine