Film tie-in Harry Potter And The Chamber Of Secrets will be one of the most sophisticated video games ever created, and the soundtrack alone has taken the designers from Surrey to Seattle — by way of a school toilet...
"Traditionally, your games sound person would be a musician who also had a great deal of mathematical skill, so that he could squeeze a lot of sound effects into a small amount of memory and probably write chip music as well," says Nick Laviers of Electronic Arts. "Nowadays, especially for us within EA Studios, it's splitting into two disciplines: there's the creative sound design side — the people generating the content — and then there's the technical sound design side, people who specialise in liasing with the programmers and actually making the sounds work in the game."
Electronic Arts is the world's biggest computer games company, and development at their UK studios in Surrey is largely focused on the multi-million pound Harry Potter franchise. Warner Bros and director Chris Columbus are currently filming the second of J K Rowling's best-selling children's stories, Harry Potter And The Chamber Of Secrets, and Nick and his colleagues are working in parallel with the film-makers on what will be one of the most expensive computer games ever written. Even though the game music is being composed out of house by Seattle-based composer Jeremy Soule, audio work in EA's purpose-built £1.6 million recording studio has involved up to 11 people.
Audio Manager for the project is Nick Laviers of Electronic Arts Studios.
As Audio Manager for the project, Nick falls squarely into the 'technical sound design' category, while his colleague Adele Kellett leads the creative team. With a background in sound design for film, it's her job to decide how the electronically recreated world of wizards and dormitories should sound, and to oversee the recording of the necessary effects, ambiences, foley and speech. It's then up to Nick to work out how this can be implemented in the game itself, making best use of the technical resources available on the various console systems for which it is being programmed. Both of them have also been responsible for finding, directing and recording the numerous child and adult actors who've contributed to the game's extensive dialogue.
Sounds & Pictures
Adele Kellett leads the creative team."The real thing that I want to do with this project is bring Hogwarts [the school in which the books are set] to life," explains Adele. "We want to make it a cinematic experience audio-wise — when you go to a film, even though it might be subliminal, the sound effects really help the emotion of the film. That's what I really want to create with the music and everything else.
"The artists started months before we did. They created a folio of all the environments there were going to be, and they found extracts from the book where sound was mentioned. J K Rowling actually puts a lot of audio cues in her writing, which is really handy. For instance, in the book, she describes [Hogwarts' kindly headmaster] Dumbledore's office in such depth that she mentions everything that you hear in that room. The portraits talk to you in Harry Potter land, so you could have the snoring of the portraits, and then she talks about silver instruments whirring and things bubbling, so you can have the whirring sound effects of these little silver objects.
"So we start off by looking at the book, and next we think about the emotion in different environments. Sections of the game that happen in the library have to be quite tense, for instance, because you're going round avoiding prefects and things like that, so we look first of all at the book and then at the emotion of what you're trying to do. Later on in the project we look at reverbs for the different rooms."
Working Prototypes
Using the game artists' work, programmers at EA create what Nick and Adele refer to as 'prototypes'. These are effectively mock-ups of the different game levels, rendered as movies rather than playable demos, and are used as Pro Tools test-beds for music and sound effects.
"Before the music is composed, I might track-lay the prototypes and just put in music from the previous game [Harry Potter And The Philosopher's Stone, which was also composed by Jeremy Soule], or snippets of music that I think would work in that scene, and then we write down what exactly we want, and send it to the composer," exlains Adele.
"We have a big spreadsheet where Adele and I will sit down and discuss the scenes of the game and decide where we want to spot the music, and what the feel of the music should be," continues Nick. "I tend to go into very technical musical descriptions, whereas Adele will talk about things in the context of the game, and how Harry's feeling. So between us we put together a music brief for Jeremy which will have that kind of overview of what the game's doing, maybe a few notes talking about notes and beats and bars and instrumentation, and then some more notes about how we want it implemented. It might just be a simple loop, it might be part of a branching piece of music, so you might have say five ambient sections that you're randomly choosing and splicing together, or it might be something even more complicated than that — we layer things, for instance, and we spot musical sounds over a main musical track to a bar's resolution — you might spot a drum beat, or whatever. We give him as much detail as is needed for the way the music is going to be rendered technically. So all that goes into a big spreadsheet, and we tend to follow up with a 'phone call. We'll talk through it until we're sure he understands what we want him to do.
The largest of the studio rooms at EA is based around a 96-channel Euphonix console.
"Once he's understood that, he'll basically compose the music and put it on an FTP server for us, at which point we pick it up. We'll review it first of all out of context with the game, because obviously if we take one listen to it and it isn't what we want, then we'll ping it straight back to him with notes on revisions. If it sounds like it's going to fit with the scene of the game, then we'll go ahead and implement it into the game — we have a tool specifically for writing interactive music for games — and see if it works. We were working a little bit blind with it earlier on, but now a lot of the levels for the game are available. It's very difficult to get Jeremy's music fully approved early on in the project, because you never know whether it's going to fit with the visuals. It might be a great piece of music, but I won't accept it until I've seen it in combination with the visuals it's intended for. He's very quick at creating synth orchestral mock-ups, so he can turn stuff around in a day."
A lot of the music — Nick estimates around 80 percent — will eventually be recorded by a real orchestra at London's Whitfield Street Studios, while choral parts recorded in Seattle will also feature. "If we're going to go for an orchestral recording, Jeremy has about three arrangers that he uses, and a recording engineer, and a music co-ordinator," explains Nick. "There's a big difference between a MIDI orchestration and a real orchestration. If you're working in MIDI you're thinking about the sound, your ultimate goal is the sound, whereas if you're writing a piece of sheet music, your ultimate goal is the notes. So what the arranger has to do is look at the MIDI score and hear the piece, and one MIDI event might be a whole effect — it might be a whole string tremolando line, or a string effect which would take several musicians in a real orchestra. So all of that needs to be expanded out, there is quite a lot of work for the arranger to do, it isn't just getting a MIDI score and copying it. The MIDI score will be quite minimal compared to what the actual musical score is going to end up looking like."
|
It All Starts To Click
Because of the way that video games consoles stream music from disc, it's usually essential to record the music to a click track so that it can be divided into regions. This allows it to be buffered from disc on playback in musically relevant segments such as a bar or a beat. "You can basically define two types of music event," explains Nick. "You can either define a 'now' event, which is triggered by the game and stops everything to do what you tell it to do, or you can have a delayed event, which waits until the end of the bar. There's various options with those as well, like the system can actually count beats within the bar, but it's only by waiting until the end of the spliced section of audio that it can actually branch seamlessly."
"The click track's really important when you're breaking things up so that they branch, because otherwise things could slip out of time," explains Adele. "The opening scene was the only one we did without a click, because it's very romantic and happy and quite laid-back, and it's just a one-off piece that opens the whole game. It was nice to do that without a click so that the orchestra could really play with it rather than it being really straight."
This is the second studio, used for a variety of tasks including programming, editing, dubbing and mixing.
"Other than that, everything is to click really," continues Nick. "For all of the prototype music and all the mock-ups that Jeremy sends, he sends a click track as well, so I always get a stereo WAV file and a mono click track, and when I'm preparing the music for use in the game, I mute out the left channel, copy in the click track, and auto-region it so that I've got all those segments there. Before it goes into the game, all the music is split up into segments on a resolution or other. Most of the time it's a bar resolution, sometimes it's actually a beat resolution, depending on how fast you want the music to change. You're steering close to the capabilities of the playback system if you're putting things in at beat resolution, especially for a very high-tempo track. We've done that once in the game, but most of the time it's a bar.
"It can branch on any bar. You segment it into where the get-out points are, so what we call each segment of a score that we split up is a node. Once it gets to the end of the node, that's a musically sensible time to branch, which is usually when we choose to branch to another track, unless of course something absolutely world-shattering happens and the music has to change instantly. We don't tend to do that unless we absolutely can't help it, because if you do that there's a slight pause — there's a latency in the system. Most of the time when something big happens you can mask that pause with sound effects, so that it doesn't actually matter that the music's coming in slightly late, because the hit point comes from the sound effects and the music just reinforces it."
|
The Battle For Resources
Sound design and music are, of course, only one component of a video game, and with only a limited amount of CPU power, memory and disc streaming bandwidth to play with, there are complicated trade-offs to be made when it comes to sharing these resources. "If your graphics engine is using disc streaming and caching as well, then it's going to limit what the audio system can do," says Nick. "We're always getting issues where one system's nobbling another: for instance, you get memory conflicts where you get graphics data written into the audio memory. Once it gets down to that level of things, adding audio to the game becomes a very fine art. There's a lot of ins and outs, things you can do, things you can't do, ways of getting a lot of audio into a small space. With the right programmer it is possible to get a lot out of the system resources. Obviously you're always balancing that against the rest of what the game's trying to do.
"There's two different types of data compression. There's the hardware compression that's contained within the platforms themselves, and then there's our own DCT-based data compression format which we've used for some years, which we can get a better compression ratio from in smaller memory areas. We try to keep the number of streams that we run down to a minimum. I'm making a distinction here between streamed sound effects and sound effects which are just triggered off and handled by the hardware. This includes speech, two CD streams of ambience and music — probably about 10 maximum. Plus we can have up to 48 simultaneous loops or one-shot sounds in hardware. And then software-wise, we've got a software mixer which basically handles all the streams from disc."
As well as the ability to hold samples in memory and stream audio from disc, some games consoles, such as the Playstation 2, also have dedicated DSP resources for processing sound. These, along with any CPU power that can be begged back from the graphics programmers, are used to treat and adapt sounds in real time to the circumstances of the game. You can, for instance, apply different reverbs to the game sounds to reflect the environment the player finds him- or herself in.
"In the past we've used distortion algorithms and resonant low-pass filters, that was basically to do dynamic engine sounds with," explains Nick. "We had a very complex system which layered the different rev recordings from the engine in different ways so you could create envelopes for every layer, and as the revs went up it would go up through these different samples. It was also processed by distortion and a resonant low-pass filter, and you could actually chain those in any way you wanted to. That was probably about as complex as it gets: Harry Potter's simple, really. It's all about the sounds themselves — if you put good sounds in the game and trigger them, that's all you really need to do, and a lot of the sounds we're putting in are actually pre-processed with reverb because they're specifically for certain locations like a dungeon, or a classroom, and there's not much point in using the game reverb for those. It's only really for things which are going to be used everywhere that we'll be using the game reverb.
"There's actually two reverbs we can use on the Playstation 2: there's a hardware reverb, which is OK, and there's a software reverb which we would only really use when it was going to be masked. The resources that we have to run a software reverb are very, very limited — you can have a multi-tap delay line, or three multi-tap delay lines, and basically that's your reverb. Every delay line needs a buffer, and the more delay lines you have, the better the reverb sounds, but you run out of resources. The hardware reverb is a slightly better algorithm — it does use buffering space, but it doesn't need as much space to get a similar quality reverb, but the problem is that for technical reasons we can only use the hardware reverb on the 48 samples that are held in hardware."
New Surroundings
Screen shots from the EA Playstation 2 version of Harry Potter And The Chamber Of Secrets.Next-generation consoles such as the Playstation 2 also support the reproduction of surround sound, but EA have decided against implementing surround music or effects in this release. "It adds complexity to the project, and there's a limited number of people who have that playback system, so we're trying to focus our efforts on what's going to have the most coverage and the most impact," explains Nick. "The most coverage is going to be people with a crappy mono TV speaker, and then of course many people will have a stereo system, so everything's pretty much stereo. We were looking at doing DTS, and we've just decided that we can't do it properly for this iteration of the game, so we're going to wait until next year when we can do a really good job on it, and more people will have taken up the hardware. We'd rather use the additional resources for other stuff. For instance, Adele's happy now because we can stream two sets of sound effects. We can have two ambient streams running simultaneously, whereas with the DTS we'd only have been able to have one, because the other ambient stream would be taken up basically with surround parts of the ambience all the time."
Speaking Parts
A visit to EA's headquarters brings home the sheer scale on which video games companies now operate. There was a time when computer games were mainly written by a single programmer, who would also tackle all the graphics and sound: now, specialist roles are beginning to emerge even within the field of game audio. Already there seems to be a clear distinction between the creative and technical aspects of game sound design, and Adele clearly expects further specialisations to develop: "Speech is becoming a discipline in this business. In the film industry, people are dialogue editors and pride themselves on being dialogue editors. There's a skill in editing dialogue to make it sound great. It's not just a case of getting the speech and going 'Hack, hack, there's your sound file.' I don't think people tend to look at it enough."
"I think if we had one person in house who was a very adept director and also had a good handle on the technical side of speech production as well, that would be invaluable," agrees Nick. "Having one director to own the whole project and have a vision for the project and be able to concentrate totally on directing the speech and making sure that's perfect would be great. If we'd had that person this year, things would have run much more smoothly. We had no conception of how big it was going to be when we embarked upon doing it."