Celemony Melodyne 3

The Editor window with the Audio-to-MIDI and Notation displays switched on. Note the continuous pitch curve that passes through all the note 'blobs'.

Celemony's Melodyne has always worked miracles with the pitch and timing of monophonic audio, but the new version 3 turns its attention to full mixes.

When Melodyne was first released back in 2001, it caused quite a stir, thanks to its ability to perform extreme pitch and time-stretching of monophonic audio recordings with a minimum of audible artifacts. Version 2 of the software, reviewed in the January 2004 issue of SOS (www.soundonsound.com/sos/ jan04/articles/melodyne2.htm), brought many improvements, including better integration with mainstream sequencers via the Melodyne Bridge plug-in and Rewire support. However, despite the jaw-dropping capabilities of Melodyne, and the fact that it is much more than a tool for pitch correction, it hasn't quite become an industry standard in the way that Antares' Auto-Tune has. In one sense this is understandable, as Auto-Tune offers an automated quick fix for vocal intonation problems, which might be all that is required by many studio owners or recording musicians. Yet there can be few users who have experienced Melodyne (even in demo form) and not been amazed at just how malleable it makes audio recordings — this is not just a 'me too!' pitch correction tool.

Indeed, while Melodyne is perfectly capable of automatic pitch correction, by allowing the user to modify the pitch and timing of individual notes within a monophonic sung or instrumental phrase, it can also completely rephrase a melodic audio recording and, as a result, also offers double-tracking and harmony part-generation capabilities that are more akin to TC-Helicon's Voice One or Voice Pro.

Despite the software's extraordinary functionality, in reviewing version 2 of Melodyne, Paul White did identify a few areas where things could be improved. Most notably, these included a need to make the workflow more efficient, and key improvements in version 3 do include many alterations to the user interface. However, perhaps the most significant additional feature within the top-of-the-range 'Studio' edition of Melodyne is the ability to work with polyphonic files (for example, a complete mix). While this doesn't offer the same degree of control as for monophonic performances, Celemony do claim that the pitch and tempo-shifting is of a very high quality.

Getting Started

The dual Mac/PC installation CD and printed manual are accompanied by a second CD that contains some 130MB of ' Melodyne ready' samples from Ueberschall, who have collaborated with Celemony on the Liquid Instruments range of software instruments. Installation proved painless on my PC test system: as well as the stand-alone program, Rewire support and the Melodyne Bridge plug-in in VST, RTAS, DXi or AU formats can be installed. On-line authorisation can be completed within a few minutes, or you can register by post, and as it is the user that is being granted the licence and not the computer, Celemony allow users to apply for a new licence key free of charge if they move Melodyne to a new computer.

Above: With the Pitch tool selected, Melodyne displays blue boxes suggesting where the notes should be centred if they were to be in perfect pitch (in this case, the performance was slightly flat for many of the notes).The Mixer window doesn't look terribly inspiring, but it offers support for plug-in effects and instruments, as with Halion shown here.At first glance, the basic appearance of the two key Melodyne windows — the Arranger and Editor windows — is pretty similar to version 2. As before, the Arranger window shares some of the general features of similar windows within a multitrack audio sequencer, and in principle, by using the rather bland-looking but functional Mixer window, it is possible to perform a full audio arranging and mixing job solely within Melodyne. That said, for all but the most simple of projects, and given the Rewire and plug-in support, I suspect most users would prefer to link Melodyne with their sequencer of choice and mix from there. To this end, audio files can be opened directly onto a track or transmitted to tracks from a host sequencer via the Melodyne Bridge plug-in.

Double-clicking on one of the track panels down the left edge of the Arranger window opens that track within the Editor window, and it is here that most of Melodyne 's magic is accessed. Individual notes are shown as 'blobs' of audio (and this is the technical term for them as used in the manual!), with vertical lines separating individual notes and the size of the blob indicating the note length and amplitude. The notes appear on a bar/beat pattern and the vertical position of the blob indicates its pitch, with notes listed on the left side of the window. A continuous line is also drawn through the series of blobs showing how the pitch varies both within a note (for vibrato or pitch drift) and at the transitions between notes (for example, the 'scoop' that is characteristic of many singers). The user can also toggle on and off a notation display at the top of the Editor window.

It is within the Arranger and Editor windows that some of the more significant workflow changes have been made. For example, when a new audio file is opened, the analysis of its pitch and tempo information now takes place automatically, rather than having to be initiated by the user. This speeds up the workflow, and even with longer files such as a complete lead vocal part, processing now seems more efficient, as this initial analysis takes only a few seconds. One decision Melodyne has to make at this point is which of its three modes to use: Melodic, Percussive or Polyphonic. Their respective functions are obvious, but on the very rare occasions when the software gets it wrong, the user can re-analyse the file via the Definition menu. As with version 2, the result of the analysis process is held within an 'analysis' file, automatically created in the same folder and with the same name as the audio file, but with a '.mdd' extension. If you want even greater control of the analysis process, the MDD Editor window (which looks very similar to the Editor window) provides plenty of further scope.

One of the key improvements within the Editor window is the new 'combi tools' feature. What this means is that the function of the most commonly used tools automatically changes depending upon where they are held over an audio blob. For example, holding the main 'arrow' tool over the centre of a blob turns it automatically into the Pitch tool, while holding it over the start or end of a note turns it into the Time tool for adjusting the start or length of the note. These sorts of subtle changes to the user interface — of which there are a good number — do bring about considerable improvements in efficiency and workflow. Aside from the new Polyphonic mode, the other improvements include reworkings of both the Rewire and Melodyne Bridge plug-ins, a new Autostretch option in the Transport Bar for easy adjustment of audio tempo, new zoom shortcuts in the Arranger and Editor windows, and menu items for automatic pitch and tempo correction.

Fever Pitch

Melodyne will appeal to many users for its creative pitch manipulation and, to my ears at least, its capabilities in this regard are in a league of their own. The various changes to the user interface and the menu structures also make these tasks easier than in earlier versions. The ability to simply drag individual notes to the required pitch, adjust the pitch transitions between notes and dampen or enhance the pitch variability of a note is, quite simply, staggering the first time you see (and hear) it being done. The results are equally good with vocal or instrumental melodies — all those stale-sounding melody loops in your sample collection can now be reshaped to provide considerably more variation. Of course, it is possible to go too far and the processing then becomes audible, but the limits are far wider than you might expect and, for doubling or harmony part generation, the quality of the results is very high.

The MDD Editor window allows detailed control of Melodyne's pitch and timing recognition process for a particular file, including specifying a specific scale.When creating such harmony or double parts, Melodyne can add random pitch and timing variations for added realism. In addition, the ability to 'snap' to a particular scale makes it easy to shift a melody line up by a certain interval and know that the shift will be dealt with intelligently. Using the Formant tool allows the formant content of each note to be adjusted independently of the pitched content, and this can be useful either to improve the quality of an extreme pitch-shift or to gently change the male/female character of the voice.

With both melodic and percussive audio files, Melodyne 's tempo and timing manipulation also provides very detailed control. Audio can be quantised to tighten up the timing or notes can simply be repositioned or shortened/lengthened. In doing the latter, Melodyne automatically adjusts the length of the adjacent note to preserve the monophonic nature of the audio. While this can generate some very bizarre vocal phrasing, used carefully, it can also rescue an otherwise good take if a minor timing problem is discovered after the singer has left the studio. It is also incredible how much a note can be stretched in length without it sounding unnatural.

Poly Power

The Melodyne Bridge plug-in — seen here operating within Cubase SX — provides a way of transmitting audio data to a Melodyne track such that any Melodyne processing is then applied to subsequent playback of the same audio.In experimenting with the new Polyphonic mode, two things rapidly became obvious. First, even when working with a full pop mix that contains vocals, guitars, key, bass and drums, Melodyne 's tempo-adjustment algorithm can manage pretty extreme adjustments before the quality starts to seriously decline. For example, I was able to take a mix originally made at 103bpm and play it back at anywhere between 85 and 150 bpm and still be convinced by the performance, even of the lead vocal. Things are perhaps slightly less flexible in the case of pitch-shifting. Shifting the same test mix plus or minus two semitones worked fine, but going any further than this, the lack of formant control produced noticeable changes in the character of the vocals and drums (particularly the snare). This issue aside, through a range of plus or minus four semitones the audio quality is fine — your singer just undergoes a certain amount of gender change. When used for manipulating a single instrument playing chords (for example, a strummed guitar loop), things can be pushed a little further, although eventually the processing does become apparent. Such limitations are, however, present in all pitch/tempo manipulation algorithms and the quality of the results obtained with Melodyne are better than any other software or hardware I've used for this task.

As in version 2, Melodyne 's Mixer can host VST and Audio Units plug-ins. There is also the ability to create 'instrument' channels within the Mixer for software synths or samplers, and this is useful if you want to experiment with Melodyne 's audio-to-MIDI conversion capabilities. While you shouldn't expect miracles, this can work really well for taking a melodic phrase and replaying it using a synthetic sound. Turning a sung vocal into a realistic violin solo with just a couple of clicks is, however, still a bit ambitious; some MIDI editing and adjustment of the pitch-bend response is generally required to make this work acceptably. The MIDI generated can be exported for use within your main sequencer if required.

Melodyne's Arranger window and transport bar.

Conclusions

While the top-of-the-range Studio version of Melodyne is not cheap, it is capable of jaw-dropping results. Celemony have certainly made progress in this release in terms of streamlining the user interface, reducing some key tasks to a single click. However, other functions do still require some level of user input, and it will be interesting to see if Celemony choose to take this streamlining process further. One obvious job that could be made easier would be harmony generation. Given that the musical key can be defined, the ability to automatically generate various harmony styles from a series of presets (as is already possible in the TC-Helicon Voice Pro) would certainly speed up Melodyne as a tool for backing-vocal production. The difficult bit — high-quality pitch-correction — already exists, and one would imagine that implementing an intelligent harmony-generation engine would be a modest task by comparison.

Even with the new polyphonic processing capabilities added in this release, regular users of Melodyne will find that the evolutionary improvements to the user interface are a bigger deal than any promises of revolutionary developments. But Melodyne, despite having now been around for five years, is still a revolutionary product. If you have never experienced what it can do, then download the demo and spend some time experimenting — and prepare yourself to be amazed as you defy audio gravity. And if the price of Melodyne Studio is beyond your particular budget, the single-track Uno and eight-track Cre8 versions provide access to the same technology at lower prices — although do check the differences in functionality before making a final choice.