Artistically Speaking
As well as the Centre and Sub channels, there's the thorny subject of how to use the rear (Surround) channels in an artistically meaningful way. Some of the full-on surround enthusiasts we've spoken to relish the thought of having backing vocals and guitar solos coming from all corners of the room, but in my view, it depends on the type of music you're creating. If you're trying to produce the illusion of a band playing on stage, then it may be enough to use surround's capabilities to widen the stereo mix and to place some room ambience in the rear speakers. Certainly this 'safe' approach provides the maximum insurance against your mix being ruined by a badly set-up playback system, but is it placing too many restrictions on what is really a very open-ended format? For example, I can certainly see the artistic attraction in using surround more overtly for ambient or dance music, and the traditionally experimental bands such as Pink Floyd (who originally did a quadraphonic mix of Dark Side of the Moon) could have a field day with their creative links. I can also envisage having a lot of fun with surround delay effects that move around the speakers.
Rik Ede is quite unequivocal about using the Surround format to the full extent of its potential, and is not so interested in producing mixes that retain stereo compatibility. "There are some really poor 5.1 music mixes where the mix doesn't take advantage of the format. I've heard live concert DVDs where only the crowd and the ambience are in the Surround channels and the band sound is just a stereo mix spread across the front Left and Right speakers. To me, that seems to be missing the point. There are plenty of 5.1 studio mixes, too, which are more or less just a traditional stereo mix with a bit of reverb going to the rear speakers. I think that's such a waste! We've yet to see much 5.1 music that's been created for the format and exploits the 5.1 palette. I look forward to the time when major artists like Madonna produce music conceived for 5.1 from the ground up. Properly used, surround can really enhance all sorts of music, including ambient material and even classical music. I did a 5.1 mix of Vivaldi's Four Seasons and the conductor said that was the first time he'd listened to a recording that sounded the way he heard it while conducting."
Unsurprisingly, Rik is fairly experimental when mixing his own material in surround, making no attempt, as some engineers do, to recreate the sound and realistic placements of a notional 'live band' around the listener. So what mix advice can he pass on? "There are no established rules in surround yet, so I'll tend to do things like keeping the snare drum in the Centre channel and the reverb from it in the Surround channels. It isn't right or wrong, but when I paint my surround picture, it seems to work for me. It's definitely not just a wall of sound at the front with reverb and delay in the rears, though.
"For a typical pop mix, I pretty much always put vocals and lead guitar in the front Centre channel. I tend to spread drums around the soundfield, too, so although I nearly always have the snare drum in the Centre channel, I'll put the hi-hat sounds in the front Left and Right channels and the bass drum about two-thirds towards the front of the mix, so a little of it is coming out of the rear Surrounds as well — I tend not to feed kick drum to the Centre speaker because it seems to sit better in the mix without it. I pan toms all around the speakers and use two sets of drum overheads, so that I can route one to the front and one to the back. Other percussion might go into the rears as well as delays from other percussion where the dry sound is mixed to the front. I know this is subjective, but being able to place things around the room can really enhance the material. Putting a small delay between the front and back speakers on a pad sound can make a real difference, and backing vocals sound great in the rears.
"It works a treat on acoustic and electric guitars, too. It's common to double-track guitars, but I've been quad-tracking them and placing them around the speakers, and it really adds to the whole experience."
Rik even likes to introduce surround effects to enhance the arrangement of a song. "On some of the surround projects I've done, I've tried to create contrast by having the soundstage quite narrow when the song starts, then introducing harmony vocal parts coming out of all the speakers when you hit the chorus."
So is there anything Rik does shy away from trying in his surround mixes? "There is the danger that some people will think it's early Beatles time again and put each instrument on a different speaker, but the novelty of that wears off pretty quickly. It's the same with extreme panning effects — moving things around the surround field can actually make you feel disorientated. A bit of subtle movement is fine, but a spinning lead guitar is overkill."
Balancing The Bit Budget: Data Compression & Fold-downs
As Peter Cobbin explains elsewhere in this article, producing a suitable surround mix isn't just a question of creating listenable six-channel audio — there are often various commercial pressures that need to be considered. Sometimes, he is asked to provide a new stereo mix of an existing track as well as a surround mix, so that both can be included on a DVD, and end users can determine whether they listen in surround or stereo. Sometimes, however, the record company choose not to fund a stereo mix, or it is decided that there simply isn't room on the disc for both, and Peter then needs to make sure that the Surround mix he makes will fold down reasonably well into stereo if downmixed by the DVD player. Abbey Road Interactive, who are responsible for DVD authoring, are the department who have to decide what rate of data compression should be applied to the various video and audio streams (for more on the various compression formats, refer back to part four of this series), and how much space in total should be devoted to the audio, video and multimedia content in any given DVD project. This is what Rob Pinniger, Technical Manager of Abbey Road Interactive, jokingly refers to as "the bit budget".
Rob Pinniger, Technical Manager, Abbey Road Interactive."There often isn't enough space for everything, or indeed bandwidth. The maximum data transfer rate from a DVD is 10.08 Megabits per second, and we never go above 9.5Mbits per second, because experience tells us some DVD players cannot play back the full bandwidth... Then you have to subtract the data rate of every soundtrack as well, even though only one will be used at a time. A DTS surround track is something like 1.5Mbits per second — although there is a half bit-rate version of that. AC3 Dolby Digital would be 448 kilobits per second, and the stereo Dolby track is 192 or 224kbits per second... That leaves a maximum video data rate of maybe 7Mbits per second, which is very poor — we would want that rate as an average. The problem is worse if the video is of poor quality as well, because the worse the video, the higher the bit rate needed to encode it accurately and avoid blocks appearing in the image. So when people come along from the audio side and say 'We want DTS and Dolby Digital, and a stereo track...', if the video master is not brilliant, we have to make decisions about cutting programme length or dropping the stereo track just to make it all fit.
"Having said that, we always try to provide a separate stereo mix for DVD‑V, because we haven't been that impressed with the automatic surround-stereo folddown. Even if the stereo mix is data-compressed Dolby Digital, at least it is always mixed specifically to sound good in stereo. We try to encode it in uncompressed PCM if we can, although again, space on the disc does not always permit that. Because most of our DVD‑V work is with music material, pop concerts and the like, we feel a good-quality stereo track is important.
"With DVD‑Audio discs, this is less of a problem, as there is no video taking up all the room. Nevertheless, we still try to fit on a surround version and a stereo version, both in MLP format. A knock-on effect of using MLP is that we can provide some downmix coefficients in the metadata which give much more control on the fold-down than is possible with Dolby Digital or DTS on DVD‑V. So we may not need to continue providing the separate stereo mix on DVD‑A in the future."
Unsurprisingly for an audio engineer, Peter Cobbin prefers to have a separate stereo mix for his projects to ensure the best possible reproduction irrespective of the playback system used, but this doesn't mean he neglects to check how his surround mixes will fold down to stereo. "I do push for a separate budget to do a stereo mix where I can. In my experience fold-downs work well for classical orchestral music and film scores, but I really believe in maximising both formats for pop music. However, regardless of whether I have the luxury of doing separate stereo and surround mixes or not, I always have the monitoring set up so I can check the stereo fold-down. Some players will inevitably fold down from the 5.1 tracks anyway, even if there is a separate stereo mix on the disc, so it is always worth checking to make sure the folddown sounds decent. To do this, I use a little Mackie mixer to manually create a folddown mix from the 5.1 surround output of the main desk. I route the 5.1 Centre channel to the Left and Right outputs at -3dB, with the rear Surrounds going to the Left and Right at -6dB, and the Sub at -10dB. I run the stereo output to an external monitor input on the SSL desk, and then I can easily check the fold-down and compensate for anything that isn't working as well as I want.
"When I'm building up a surround mix and sending signals from the front to the Surround channels for ambient effects, I get better results by delaying the signal to the rear rather than using the direct signal, and I might also have the delay feeding a separate reverb. Depending on how long or short that delay is, sometimes when you fold the rears back into a stereo fold-down it sounds great, although sometimes it sounds like I've applied a comb filter! In those situations, I tweak the delays to make the fold-down work as well as possible without compromising the surround mix too much."
Peter views the various data-compression methods used on DVDs with equanimity. On the one hand, data compression affects the sound of his lovingly crafted mixes — but without it, there wouldn't be much hope of fitting video and audio of any great length onto DVD at all. "Needless to say, I think that the uncompressed masters sound better, but I still think AC3 and DTS are great formats. When I did my first mixes for DVD, I got AC3 and DTS encoders in so I could hear what they were doing as I mixed through them. I think it's fair to say most engineers prefer the sound of DTS. However, AC3 is in the spec of DVD as a mandatory format, and I've learned to live with it. These days, I don't unnecessarily modify my mixes for the sake of the compression format, which may change in the future anyway. Instead, I concentrate on producing the best 24-bit, 5.1 surround tracks I can, which may then be used for DVD‑V, DVD‑A or whatever format is deemed appropriate."
5.1 Remixes
Of course, Rik has the advantage of working on self-composed original material most of the time, which means he can give his imagination full rein when he wants to experiment. For engineers like Peter Cobbin at Abbey Road and the well-known American producer Elliot Scheiner, who are currently remixing record company back-catalogue material into 5.1 for release on DVD (Scheiner has worked on 5.1 mixes of The Eagles' Hotel California and is preparing a mix of Queen's 'Bohemian Rhapsody' as this is being written), there are additional complications. For a start, if you're working on a soundtrack which is destined to accompany a video on DVD, as Peter Cobbin often is, you may feel restricted in what you can do to the audio because you have to reflect what's happening on screen.
Sometimes, multitrack tapes of the original recordings may no longer be available — or the record company may not be prepared to make them available for budgetary reasons! In such cases, you may have to work from a stereo mixdown, rather than the original multitrack master. The surround remix process is then known as an 'upmix' (see 'Practical Upmixing' box at the end. The technical side of the process was described in Part 5 of this series). Finally, there are what you might call ethical considerations — when remixing for surround, the music you're working with is often well known in an existing stereo mix. How closely should you follow this version, and to what extent should you use surround effects? There may even be a brief from the record company or artist which restricts what you can do.
As Senior Recording Engineer at Abbey Road Studios, Peter Cobbin is familiar with taking points like this into consideration before he even starts work on a given project. Over time, he has become accustomed to getting a wide variety of guidance from record companies, ranging from nothing at all to an exacting list of what he can and can't do. "I have been briefed to 'make it sound exactly like the stereo', but of course that's impossible, as they actually want it in 5.1! I usually take that to mean that they want the same effects, delays, EQs, and so on to evoke the same sort of feeling that you remember from the stereo, but for it to have the experience of 5.1. Sometimes, people are less guarded and say 'just do whatever you need to do'.
"There aren't any rules on where to begin with a surround mix. Sometimes, if I've been asked to do a new stereo mix for the DVD as well as the surround, I'll do the stereo mix first, as that helps me get to know the track, and work out which things need particular attention. Then I go on and build the surround version afterwards. But I'm not always given the budget to do a separate stereo mix."
When mixing surround soundtracks for DVDs with video content, Peter pays careful attention to the positions of sound sources, and does his best to ensure that there are no discrepancies between what you hear on the soundtrack and what you see on the screen. "I tend to keep sounds more towards the front when working with pictures than on a pure music track, particularly if the music involves a singer. I think it can be very distracting if you can see the singer in the middle of the screen, but the voice is coming out all over the place!"
Sometimes, though, certain aspects are completely beyond Peter's control. When taken on to remix the soundtrack for a DVD of Freddie Mercury singing his 1988 hit 'Barcelona' live in concert with Montserrat Caballé, he was given not the multitracks of the version recorded on the night of the concert, but the multitracks of the studio version of the song, and told to use those! "I was told to make the soundtrack feel like the video looks — that is, a live concert on a big open-air stage. But then I was sent these four 24-track analogue multitrack tapes recorded in the studio. They were very dead-sounding, and very unlike what we were seeing in the pictures, which showed fireworks going off and the audience applauding at the climax of the song, for example. None of that was on the studio tapes, of course!
"That wasn't the end of it, either. I don't know the full story, but apparently at the concert there was a Nagra tape machine playing the backing track of the song, with live vocals from Freddie, Montserrat, and backing singers. The thing was, for some reason the Nagra played back a bit slowly on the night. Freddie, being the consummate pro, held the performance together, but this meant that I then had to sync the studio version over a slow performance! The record company didn't want to have the film recut, so I just had to do the best I could. The lip sync is not particularly wonderful as a result, but I couldn't do much about it!
"I had to create all the live ambience, which I did by playing the studio track out through speakers into Studio 1 and miking them up with a surround mic array tie-lined back to the desk. I used foley samples for the fireworks and audience applause at the end. I layered various different audience samples around the surround soundstage, and applied lots of different reverbs to help build up the signal in the Surround channels.
"This is one of the things that I have learned about working in surround; often the surround mixing is just one component. I may end up mixing a whole concert, adding sound effects, laying it all back to picture, taking care of the fold-down stereo and even the mastering — all under the banner of 'remixing for surround'."
Much less difficult were the production of DVD soundtracks for a live Eurythmics concert and a 5.1-compatible version of John Lennon's 'Imagine' to accompany Gimme Some Truth, a DVD about the making of the original album. In both cases, Peter was given complete artistic control. Nor was the raw material problematic as with the Freddie Mercury DVD. The Eurythmics concert had been conceived as a 5.1 DVD‑V from the beginning, and had therefore been carefully miked and recorded to multitracks on the night. In the case of 'Imagine', Yoko Ono, impressed by Peter's work on the surround soundtrack for The Beatles' film cartoon Yellow Submarine, sent him a copy of the original eight-track multitrack master! Peter approached both projects differently. For the Eurythmics concert, he once again used the facilities at Abbey Road to recreate the feel of the concert. "These concerts involve a lot of close-miking and DI'd instruments, so you don't get any sense of ambience from those. There were special audience mics set out, but even when I put those in, I felt it was missing something. I've found this even with stereo: sometimes to create a live-audience effect it has to be larger than life, so again, when I mixed this I had a lot of things pumping out into the studio and being re-recorded. I had a signal going out through a guitar amp, there was a Leslie speaker going at the back of the studio, and the main speakers were carrying something else again — you could walk around in the studio and it was almost like you were at the concert!
"Now you can really hear the interaction with the audience, which is nice, and I tried to create a sense of the depth and the size of auditorium in the surround soundstage, too, which the original multitracks didn't have. Now, when Dave Stewart plays a guitar chord, it seems to bounce off the back wall — all that kind of thing I had to create with processing and re-recording."
Peter took a more conservative approach with the mix for Lennon's 'Imagine', although it's still impressive when you hear it over a surround system. The main 'rock' instruments — bass, piano, and vocal — are spread predominantly across the front three speakers, and the strings, which enter at the start of the second verse, are mainly routed to the rear Surround channels, enabling you to hear the string arrangement with a clarity impossible in the stereo version.
In Part 8...
As this article has shown, there are currently plenty of questions about surround production, but no hard-and-fast rights and wrongs, so the only useful way to find out more about creating surround sound is to talk to people working with it! With this in mind, next month, we'll get the low-down on how two more producers are coping with working in 5.1 — Gavin Sutherland, a folk musician who has set up Scotland's first surround-capable project studio, and the aforementioned Elliot Scheiner, who should have wrapped up work on his 5.1 mix of 'Bohemian Rhapsody' by then...
Surround Sound Explained: Part 1 Foundations
Surround Sound Explained: Part 2 Dolby Pro Logic
Surround Sound Explained: Part 3 Ambisonics
Surround Sound Explained: Part 4 5.1 Surround
Surround Sound Explained: Part 5 Metadata, Upmixing, Downmixing & The Centre Speaker
Surround Sound Explained: Part 6 Setting Up A Surround Recording System
Surround Sound Explained: Part 7 Mixing In Surround
Surround Sound Explained: Part 8 Surround Production
Surround Sound Explained: Part 9 Surround In Your DAW
Practical Upmixing: Converting Stereo to 5.1
Andrew Walter is one of Abbey Road's 5.1 Audio Restoration engineers. Despite his title, he is not only responsible for the restoration of damaged or noisy recordings, but also mastering, AC3 and DTS encoding of surround material, and upmixing — the production of 5.1-compatible mixes from stereo (or even mono) sources. At the time of SOS's visit, Andrew was creating upmixes from a bizarre range of source material, including a pile of shellac 78rpm records, some mono piano concertos recorded when magnetic tape was a recent invention, and, bizarrely, a new Emma Bunton single. Upmixes of the first two of these were understandable, as there never was a multitrack version of the material available, but we were puzzled as to why the Emma Bunton material was coming to Andrew in stereo instead of going to Peter Cobbin in Studio 3 on multitrack for a 'proper' surround remix. It turns out that the reason is purely economic. Upmixing, as explained in Part 5 of this series, is a relatively simple process, and can be done using a finished, paid-for stereo mix as a starting point. Put bluntly, it takes the likes of Peter Cobbin a lot longer to produce a true, discrete-channel 5.1 surround mix (and therefore costs the record company a lot more) than it does to pay Andrew Walter to create an upmix from the stereo version in a few hours. But as he explains, it's not quite as crude as that...
"It's not always best to go back to a multitrack source for surround mixes. Very few multitracks were recorded with any kind of surround sound in mind — most were intended for stereo at best. So even when you have the multitrack, there's always an element of fudging things to create surround ambience because it wasn't recorded with that in mind. You don't find many tapes with ambience tracks ready-recorded for the Surround channels, believe me. So even engineers like Peter have to make that up during the remix. Of course, what they can do better from the multitrack is produce a decent spread of individual sounds across the front three speakers, send things deliberately to the sub, and pan solo instruments — but the ambience sound will still have to be faked!
"It also depends what you're mixing. I'm upmixing a live Deep Purple concert for DVD at the moment, and when you watch the video, your perspective is that you're in the audience listening, so you don't really want to have separate instruments out of the mix and flying around the soundstage — it wouldn't be realistic.
"We've tried various test recordings for EMI Classics in here, and some of those were originally recorded to multiple channels with quad sound in mind. Those we did remaster in four-channel surround sound, and it worked very well. Most of the classical recordings, though, were stereo, and we could have remixed those to surround in a variety of different ways. At the time, we chose not to use technology, but achieved the surround ambience by playing the stereo tracks out into Studio 1 and miking it up. That's the room where the recordings were originally made, so it was very effective. However, now we have got to grips with our new TC System 6000 surround processor, I'm not sure we'd do it like that again. If you know what you want to create and you're careful, you can do that with good digital reverb and delays. We have the TC set up so that one DSP engine provides 5.1 panning capabilities, and then we feed that into a second Engine to introduce multiple delays and some sort of reverb effect.
"The System 6000 now offers an 'Unwrap' algorithm, which is supposed to automatically convert stereo to 5.1, but I haven't been that impressed with it so far. I'm sure it has its uses if you need to put something into 5.1 really quickly, but you can't really control anything sufficiently for our purposes."
So how do Andrew and his colleagues perform an upmix? "We take the left-right stereo mix and always keep that exactly the same in the Left and Right channels of the surround mix. Then we add to it a Centre, Surround and Sub channels as necessary, so we are not damaging or destroying the original sound, we are just adding to it to enhance it for surround.
"To take a specific recent example, we took the original stereo, put delays on it and sent it to the surrounds with an effect and a stereo-width enhancer on it to widen it. The problem is always how to fill in the sides, and how to create the Centre channel — what can you really do other than put a mono sum at a lesser level in the centre? On some things I send some of the information going to the sub directly into both Surround speakers, so that it helps focus the bass on a line running from front to back. I also often shelve off the high end going to the Surrounds.
"In a nutshell, what I like to hear from an upmix — a classical one, anyway — is really just an enhanced stereo where you hear a few more reflections from behind in the Surround channels. Ideally, you have hardly anything coming out, and we do tend to err on the side of putting very little there. We did some tests where we played material to the other engineers in the building and many of them questioned whether the Surround speakers were on at all, but after they had listened to it for a while we switched to stereo, and then they realised just how important the surrounds had been in creating the illusion of space and ambience. I think it's nice to use it very subtly like that.
"On the other hand, with pop music, there is more freedom to use everything less subtly. With the Emma Bunton single I've just done, the Sub comes in and out at different places, and the Surrounds are not used so much during the verses, but then come in heavily on the choruses for emphasis.
"Obviously, as an engineer, the more options you have, the better, so if you have multitrack tapes recorded with surround in mind in the first place, that is probably the ideal material to start from. With something like the Eurythmics' Peacetour concert Peter Cobbin worked on, where they had the luxury of being able to spend the thousands of pounds needed to go back and remix it in full surround, that was fantastic. But whether the market will ever allow that to happen on a broader scale is another question. We know people in America who are setting up purely to provide upmixes from stereo material for surround release. I think if DVD‑A takes off and people want surround, there's a good chance that most of the 5.1 mixes will be produced by upmixing.
"The bottom line is that if you don't like what we have added in the upmix, you can always get back to the original stereo. We have found, though, that in most cases people don't realise how it has been done, whether it has been upmixed or rebuilt from original multitracks." Hugh Robjohns