You are here

Designer Sounds

Sound Design For Visual Media & Radio
By Nick Metcalfe

In the second part of this exploration of the craft of sound design for visual media, we reveal the skills needed to make good music edits.

It almost goes without saying that music plays a crucial role in sound design. While we may each have different personal tastes, we can all succumb to the power of music when we watch a film or TV programme, or listen to a radio drama. It holds the power to engage our emotions, and enriches our experience of the film, game or whatever it is we're immersing ourselves in.

As a good score weaves in and out of the action, reinforcing the dramatic highs and lows, we find ourselves empathising more with the drama in the story, the plight of the characters, and the underpinning emotional intent of the piece, than if the scene were devoid of music. Film makers have understood for many years the effectiveness of music in selling a mood or theme, but new media such as video games are also becoming sophisticated enough now to adopt some of these cinematic techniques.

Regardless of the medium you're working in, editing and manipulating music is an important skill to develop. Thankfully, if you develop a few basic skills and maintain an attention to detail, you can learn to create effective and convincing music edits. In fact, irrespective of your knowledge of music or performance skills, a sense of rhythm, musical timing and a keen eye to make precision edits are the only really essential requirements.

Editing Or Composition?

Most large film, TV or game productions typically feature music that's been written specifically for them, but in more modest productions it's still common for the sound designer to edit existing music against visual cues (and, indeed, non-visual cues, such as the presence of dialogue or a particular sound effect). The same can apply in cases where the director has an existing piece of music in mind for the scene, or when editing promos, trailers, commercials and other shorter material.

In a few cases, you might find that the visuals have already been edited to fit the music, in a similar way to a typical music video. In such situations, the music has been chosen first, presumably because it conveys a certain style or energy, and the video editor is tasked with making his or her cuts against the rhythm or flow that's already present. This is not an uncommon approach in film trailers, for example. Note, though, that this approach is not the norm, which is why I'll focus in this article on the skills required to edit music against a final 'locked' visual edit.

Just as when composing, it's really useful to go through the visuals and create markers for significant on-screen events.Just as when composing, it's really useful to go through the visuals and create markers for significant on-screen events.

An Illusion Of Seamlessness

Regardless of the intention of your music edit, the goal is usually the same, which is to bring different sections of music together and produce an edit that will create the illusion of one flowing piece, even if that includes dramatic changes in tone or genre. With practice, and a little patience, the sound designer will become adept at choosing appropriate music, and editing and blending it against almost any visual footage, almost giving the impression that the music was composed specifically for the visual.

You can break down the music editing process into a few key stages:

1. Study The Visuals: Before going anywhere near the music, study the film. Familiarise yourself with its visual editing style, and get a grasp of the story. Then, start to determine how music can be used to good effect, by considering the story and mood that's being conveyed. It's hugely important that you don't simply add music across the whole piece if it's not required — the music must justify its existence!

Instead, aim to reinforce the main dramatic points. Take care to note just how the video editor has sequenced the piece together: does it use slow, steady shots that dissolve between cuts, or is it a more frantic music-video sort of edit, with short cuts in rapid succession? Are there particular scenes or events in the piece that would warrant a music cue, such as an explosion, a sudden contrast in the drama, or an actor's reaction to an event? Once you're familiar with the visuals and have considered the dramatic highs and lows of the piece you can move on to the next stage.

2. Choosing Music: The genre or style of music will largely be determined by the visuals and the overall creative direction of the work. The director might have given an indication of the sort of music they have in mind, in which case it's still worth talking to them before you make decisions, but others might want you to bring options to the table. Bear in mind the laws of copyright: you can't just use your favourite artist's commercial track for free if the end product is destined to be released or broadcast. Love or hate the idea, there are plenty of music libraries that sell 'stock' royalty-free music which can help you avoid that particular pitfall. The quality can be variable, though, so spend plenty of time searching for suitable music. You might have your own stock of ideas and snippets that can be used, of course, but that's a whole different article!

Finding music that fits the mood may be critically important, but there are some practical considerations too. For instance, working with contemporary rock, pop or dance music can be particularly challenging if you need to edit around the vocals within the music. It's often easier to use tracks in these genres that don't contain vocals, as you then won't need to work around the problem of editing them in incorrect sequences. Much modern music also has a restricted dynamic range, with modern pop songs, in particular, often lacking much variation in energy. This can be problematic as you'll often find that the 'natural' rise and fall in volume and instrumentation of music helps your edit to work well and gel with the picture. This isn't the place to debate the 'loudness wars', but I will point out that less-dynamic music can also be more challenging to edit, simply because there are fewer changes in amplitude that serve as natural edit points. Conversely, music made specifically for movie or television soundtracks is often easier to work with, especially for the novice sound editor, as it's usually very dynamic, with sweeping phrases that rise and fall, and which can usually be synchronised relatively easily to all but the trickiest of visuals.

Working with multiple music pieces from different genres can present another challenge. You're likely to have to work very hard to make the results flow as well as when similar pieces are used. Some visuals, however, exhibit dramatic changes, which reinforce a more extreme music change, and therefore the edit can still be effective — and may even be more appropriate.

3. Music Editing: Having studied the visuals and considered some music ideas, you can start the editing process itself by bringing the visuals into your DAW software, setting up the audio side of your project and getting to work.

First, add markers at points of interest. This will make navigation of the visuals easier, as well as highlighting potential edit points. Mark scene changes, dramatic events, logos appearing, and so on.

Next, create two stereo tracks for your music, and one for cuts and edits. (I call that one my 'working track'.) Add sections of music to your working track and perform your cuts and edits there, then add the edited sections to one of the two main music tracks. When you create another edit you can add the piece to the second music track, then next time the first again, and so on, to build a pattern of alternative music tracks per edit. This makes cross fading and blending much easier.

Now, consider your approach before you begin editing by thinking about exactly which visual or non-visual cues should be given priority for a music change, and which are less important. Since music is a linear medium, it may make sense to start at the beginning and work through to the piece's conclusion, but some sound editors prefer to work on the more important cues first. For the novice it's probably easier to work chronologically.

Finally, use the mute and solo buttons to audition your edits, without hearing what's on the working track.A music-editing session in progress, with markers where scenes commence, two music tracks, and one 'working' track for edits.A music-editing session in progress, with markers where scenes commence, two music tracks, and one 'working' track for edits.

Editing Techniques

I've used a lot of words explaining the general approach, simply because that's so important, but there's also a technical side to this discipline, so let's now look at some of the techniques required to perform successful music edits.

Accurate Edits: Good music editing requires frame-accurate and beat-accurate precision, especially when the edits themselves fall exactly on a musical beat or a visual cue. Novice sound editors learning the skills often produce work that fails to correctly connect with the visuals, with music cues hitting very slightly early or a little late. You must use your DAW's zoom and frame-by-frame scrubbing facilities to ensure your music cues hit your visual cues accurately. Even if it's two frames adrift, the edit can feel sloppy, as our brains struggle to reconcile the lack of synchronicity between sound and picture.

Familiarise yourself with the 'scrub' features of your DAW, and scrub the movie back and forth until the exact frame of change is visible. You can add a marker here if required, and the music cue can thus be positioned exactly where it's needed. You must also get used to looking at waveforms, and getting a sense of the music purely by sight. I realise that this is different from the music-making advice frequently dispensed in these pages ('just use your ears'), but this way you can match a beat, which is usually represented by a spike in the waveform, directly with the visual cue on the exact frame. It may be useful to enlarge the waveform vertically as well, so that the transients and dynamics of the waveform are more clearly visible.

Beat Edits: Often, the most effective music edits are those that occur on a beat — ie. where two pieces of music blend together but the rhythm is not interrupted, so let's look at how to perform a basic 'beat edit' between two pieces of music:

  • Take note of the rhythm of the first piece, and make a cut just after the beat in which you want to transition to the second piece. Add a tiny fade-out after the beat, maybe three frames long or less.
  • Position the second piece of music you want to edit on an adjacent track. Take notes of the beat in this piece and ensure it is already trimmed to start at the point on a beat that is desired. Slide this start beat under the end beat of the first piece and, if required, perform a small fade at the front edge in order to prevent a click as the new piece starts.
  • Experiment with the timing to create a seamless beat-edit transition between the first piece and the second. Note that beat edits tend to be especially effective when synchronised with a short visual cue, such as a quick transition or a short visual event, like an explosion or gunshot, for example.

Successful Crossfading: Sometimes it's necessary to perform an edit between slower or less beat-driven pieces of music, perhaps from one sweeping phrase to another in a piece of orchestral music, or from one atmospheric chord or drone to another. For these types of transitions your primary tool should be the crossfade. Essentially, you're fading out the first piece of music at the same time as an overlapping second piece fades in on an adjacent track. This is easier than trying to perform a crossfade with both pieces placed on the same track. Sometimes, a successful blend requires a longer crossfade to create an effective transition, possibly as long as two or three seconds. In terms of skills required, you must listen carefully to the piece and simply make a judgment; what is successful will differ, so there aren't the same 'golden rules' as with beat-match editing.

Inexperienced sound designers often attempt to crossfade material that differs wildly. The results tend to sound like an obvious 'edit'. Of course, as you become more adept you can experiment with more radical musical changes, but it's better to start by using similar material, perhaps selecting music from the same soundtrack, artist or library. Take care to make sure that the resulting edit still sounds musically correct, and that the transition between the two pieces is not too disjointed. (I've heard some terrible, yet technically seamless edits of a single track on TV adverts where a beat is dropped to shorten the piece!) Remember, your aim is to create a single flowing track made up of separate edits.

The first screen shows how a musical intro has been created. Note how one piece obviously builds and abruptly cuts off where the second, more obviously rhythmic piece takes over when the on-screen action ramps up. The second is an outro edit, which is similar in principle, but this time there's overlap between the two clips. In both cases, the lowest 'working track' has been used before copying the clips to the music tracks.The first screen shows how a musical intro has been created. Note how one piece obviously builds and abruptly cuts off where the second, more obviously rhythmic piece takes over when the on-screen action ramps up. The second is an outro edit, which is similar in principle, but this time there's overlap between the two clips. In both cases, the lowest 'working track' has been used before copying the clips to the music tracks.

Intros & Outros: The intros and outros of your music edit may be the most effective and most dramatic elements across the whole piece. Consider the music edit in a movie trailer, for example. The intro may be tied tightly to a visual cue in which the nature of the visuals is revealed to the audience. Perhaps a rising chord that hits a crescendo as a camera cut reveals a character or event. Then the trailer plays out through the middle section where we begin to learn more about the content of the movie, the relationships between characters, and the over-arching storyline or premise. The outro of the trailer is carefully constructed to leave the audience gasping for more, and the final shot usually shows the name of the movie on screen along with information about its release date. It's crucial that the music edits support this structure and lends gravitas to it.

When creating intros and outros, the sound designer often has the advantage of using the intros and outros that were already present in the music that's being manipulated to fit the visuals. It may also be necessary to use an intro from one song or piece, and then directly segue to another piece, when a suitable visual cue is present. As an exercise, grab several pieces of music and listen to the way they begin. You may notice that some pieces fade in, others start on a defined beat, some start with a simple note or drum pattern with further instrumentation added over the next few bars. Sometimes intros build quickly to a crescendo before the beat or the main body of the music takes over. You can take advantage of such intros and use them to reinforce your first one or two visual cues.

Outros can be a little trickier to work with. Sometimes we may opt to fade the music track out rather than create a specific outro part, and this choice can depend on the visual itself (does it fade to black or does it end on a striking image?). To create a powerful outro, examine the natural outros of the music you're working with, and edit these into the music at the point you want your piece to end.

A beat transition between the main body of the music and the natural outro is usually a good way to work. The outro may end on a reverberant drum hit, or climactic orchestral hit, and these types of outro are highly effective when working with visuals that end on a specific visual cue: the music will end naturally, and the illusion of the music fitting the visuals as if written for them will be complete.

Designer Sounds

Weaving & Mixing: Music is often at its most effective when it's allowed to weave in and out of the sound track, with sound effects and dialogue taking precedence in some areas, and music taking precedence in others. Rather than play throughout the piece, the music moves in and out, reinforcing drama where required — and so the music has a greater emotional impact. Sometimes, music may be present throughout a piece and still be effective, but in these cases the music itself will be dynamic enough to rise and fall, creating natural spaces for any sound effects and dialogue.

With all this in mind, it's important to take a step back from your editing, examine the visuals and look at the effectiveness of your music edit and other sound design as a whole. Are there points where you could fade the music away and reintroduce it in a further scene to increase the dramatic effect? Perhaps you could have a musical intro give way to non-musical atmospheric sounds before being reintroduced at a later point. Or maybe the music itself should not begin until the most dramatic part of the piece hits, with the preceding scenes created entirely by sound design. All such options enable you to make the music more dramatic and more effective.

When you reach a point where your music edits are complete, you can start pre-mixing the music against the other components in the piece, before the final mix where all components are considered. Remember that the role of music is to underpin the emotion in the scene. Therefore, when mixing you can usually let the music sit further back, allowing the all-important dialogue to cut through, but bring the music forward when required to embellish a visual cue.

Designer Sounds

Tips & Tricks

With those bread-and-butter techniques under your belt, let me dispense a few more tips and tricks:

  • Once music is introduced in the piece and heard by the viewer, it may be possible to pull back the volume and therefore create space for other sounds. The music will likely be in stereo and will fill the width of the piece, while mostly mono sound effects and dialogue are positioned in the mix using panning.
  • Music may need to 'duck' (be reduced in volume by a small amount) whenever dialogue is played in order to maintain intelligibility of the dialogue.
  • Pre-mix your music separately from other audio elements, making sure that a balance is made between each of the music components used, especially if they're of different styles. Creating a master mix of all the music edits merged into one file, using a bus compressor for basic, gentle, level control, can help you pre-mix the music in a way that means the volume levels of the differing pieces are usefully smoothed out.
  • Working with music can be incredibly rewarding for a sound designer, but initially it can also be frustrating as you struggle to blend some tracks coherently. If you reach a brick wall, it might be the choice of music that's causing the problem, so try looking for music that lends itself better to the visuals, and try out some different edits.

Workflow & Experimentation

Usually, when I have the responsibility for the music edit — as opposed to those projects where I work as a sound-effect designer alongside a music composer or dedicated music editor — I'll work on the music only after I've done my initial sound effects and dialogue 'passes'. This is simply because it's very easy to neglect the sound effects if the music is done first, as you'll inevitably tend to let the music carry the piece, even if it would be better served being led by sound effects with only some music support. A better workflow is to have a solid and rich sound-effect pass already in place before adding the music, which can then be 'peppered' and woven in to emphasise drama where it's needed. Using this approach, you should find that you can make more considered decisions on where the music needs to be very pronounced, and where it needs to sit back and let sound effects or dialogue come to the fore.

  • As with sound-effect design, the key to achieving great results is often experimentation. For any given project, I may create four or five different music edits and have them lying on separate tracks in my DAW software. That way I can instantly compare them alongside the visuals using my solo and mute buttons.This is also a useful exercise to perform if you're uncertain of the genre to use — should it be hard rock or hard techno music against this action scene? Try various combinations and see how they emphasise the visuals differently.    
Published November 2013