You are here

Dealing With Computer Audio Latency

PC Musician By Martin Walker
Published April 1999

Yamaha DSP Factory soundcard (reviewed SOS October '98) — one of the elite group of soundcards that provide a hardware solution to problems of latency.Yamaha DSP Factory soundcard (reviewed SOS October '98) — one of the elite group of soundcards that provide a hardware solution to problems of latency.

The small and not‑so‑small time delays between a signal entering and then emerging from a computer soundcard can turn hard disk recording into a frustrating experience. Martin Walker leads us through the twists and turns of latency.

Latency may sound like something that escaped from a physics lesson at school, but it's real, affects lots of musicians who work with computers and audio, and is extremely frustrating. Only when you experience the problem do you realise why so many people have been bleating on about it for so long.

So just what is it? When you put a new soundcard through its paces, one of the first things you are likely to do is to listen to some of the many real‑time effects plug‑ins available. You connect your mic, guitar or keyboard, patch in the effect, and then notice a half‑second delay before anything happens. So you look for the switch to turn off the delay effect. Only there isn't one. What you're actually hearing is the result of latency, and, worse still, there's no way to get rid of it. At this point, many people wish they'd bought a hardware‑based digital recorder instead, and also wonder why these effects are called real‑time when they appear to be nothing of the sort.

However, there are ways to minimise latency, though this usually involves making decisions before you choose your soundcard, not afterwards. So please read on. Finding out about latency might just prevent you from buying the wrong soundcard.

Latency — The Facts

Cubase shows the current figure for latency in its Audio System Setup window. The figure of 557mS shown here is fairly typical when using the standard Windows Multimedia drivers.Cubase shows the current figure for latency in its Audio System Setup window. The figure of 557mS shown here is fairly typical when using the standard Windows Multimedia drivers.

Latency is normally thought of as the time delay between inputting a sound and hearing it emerge from the soundcard. However, it also affects playback of existing tracks and the general responsiveness of the application as outlined in the 'Other Effects Of Latency' box elsewhere in the article. The reason that the signal can get so delayed is that it has a long way to go. First, it gets converted from analogue to digital via the converter (a matter of about a millisecond, but a delay nonetheless). Then it passes internally along a digital buss inside the soundcard, and emerges on to the PCI buss (the connector on the edge of every recent soundcard). From here it travels on the motherboard to the rest of the computer, where it meets a huge obstacle — the operating system. Whether Mac or PC, the modern operating system has a great deal to do, and so it does a little bit of everything in turn. As well as processing your audio, the operating system has to do a lot of 'housekeeping' tasks, and to keep everything tidy, it relies on the 'little but often' approach.

At this point, many people wish they'd bought a hardware‑based digital recorder instead, and also wonder why these effects are called real‑time when they appear to be nothing of the sort.

To ensure that there are no major glitches in the audio, the soundcard driver software has to be designed to anticipate and cope with any unexpected delays before the operating system deals with the audio. As is always the case with such delays, a small queue of audio data can form, and this slowly backs up in small memory buffers in the soundcard driver until the operating system returns from carrying out its other duties.

As we musicians don't like chunks missing from our audio, software developers include small RAM buffers as part of the sequencer design to hold enough audio data to ensure its smooth delivery. There will be a separate sequencer buffer (or set of buffers) for each enabled input port of your soundcard, and these get topped up by the operating system in fits and starts whenever it gets the chance to grab some more audio from the buffers on the soundcard. As long as there are enough buffers of a big enough size, there will always be enough data in hand to carry on delivering audio to the music application at a smooth and even rate.

The sequencer then carries out lots of number‑crunching — especially if you're adding software‑based effects and EQ. Finally, the digital signal is sent to another set of buffers — one for each output port of your soundcard — this time to ensure that nothing gets lost on its outward journey. The operating system comes along when it's good and ready, shunts the new digital data stream back across the motherboard, through the PCI edge connector into the soundcard buffers, and it is then clocked out steadily at the chosen sample rate. After another millisecond's worth of conversion time it emerges as an analogue signal from the soundcard output socket.

This is a long journey, the exact length of which depends on the route taken. In the case of a PC, the Windows operating system has a lot of other things to do besides updating its standard Multimedia (MME) soundcard drivers. This means that more and larger buffers are needed to hold the audio data to ensure that nothing is lost. Typically, an application like Cubase VST using the bog‑standard MME drivers will have a latency of maybe half a second, depending on your settings. This may not sound like much, but if you remember that it's the same length as a single beat at 120bpm, you can begin to understand why it's such a problem. It means that when you are recording, you can't hear what's being recorded until a beat after it happens. But most people only discover this after they buy a system and set everything up. No wonder they get frustrated!

The Solution

The PCI card from Ensoniq's PARIS system is one of the selection which has a hardware thru to eliminate latency.The PCI card from Ensoniq's PARIS system is one of the selection which has a hardware thru to eliminate latency.

What's the answer? Does every soundcard suffer from this problem? This is where things get more interesting. As most people would prefer to bypass the problems of latency altogether, I'll start there. As described above, the length of the route taken by the audio signal is largely due to the operating system, and the RAM buffers that are needed to keep the audio stream steady and glitch‑free. The way to avoid all these delays is to keep the audio signal on the soundcard for monitoring purposes, and ignore the operating system completely.

Soundcards usually contain some sort of digital mixer. This is either permanently burned into specialist DSP chips (like the Yamaha DSP Factory) or the soundcard uses generic, programmable DSP chips (more often than not by Motorola). In the latter case, the code for the digital mixer is downloaded from your hard drive before it can be used, a process which normally happens as your computer boots up. All the soundcard developer needs to do is provide a way for you to route the signal at the soundcard input socket to one or more of its output sockets — a sort of hardware thru. The route taken by the audio signal then involves merely passing through the A‑D converters (about a millisecond), going down a printed circuit board track, and then being converted back into analogue through the D‑A converter (another millisecond). And there you are — you can hear your input signal with no more delay than you get when using any outboard effects hardware. If your soundcard features multiple outputs, you can even create several different monitor mixes for individual musicians.

Musicians will be most comfortable with a figure of 10mS or less — the equivalent latency value you find on MIDI gear between pressing a key on the keyboard and hearing the sound.

Various soundcard manufacturers have adopted this solution. The Lexicon Studio provides 'Punch Record', complete with its own software‑controlled mix level fader, which allows you to bypass the normal monitoring facilities inside applications like Cubase, and directly patch any selected combination of Lexicon Studio input signals through to one of the Lexicon Studio hardware outputs during recording. The MOTU 2408 has its CueMix Console utility, and the Event soundcard range has the Echo Console. The Yamaha DSP Factory has an extremely comprehensive digital mixer offering hardware thru as just one of its many features.

It can be confusing when you launch one of these routing and level control utilities, since many people assume that if it appears on a computer screen then the mechanics are no different from an application like Cubase VST. However, there is actually a big distinction between the two. Audio software such as Cubase VST runs entirely in software, and is therefore subject to operating system delays. The soundcard mixer utility, while it may be controlled using software, is actually a remote control for the DSP‑based routing and level controls on the soundcard itself. Although changing the position of any control will be subject to operating system delays, the audio monitoring signals themselves never leave the soundcard, and are therefore entirely independent of the operating system.

In Full Effect

If your soundcard provides remote control of its hardware features then you can bypass the high latency software route altogether. This is the Echo Console (used by Event's Darla, Gina and Layla soundcards). It provides comprehensive routing and level controls for inputs and outputs, along with fast metering.If your soundcard provides remote control of its hardware features then you can bypass the high latency software route altogether. This is the Echo Console (used by Event's Darla, Gina and Layla soundcards). It provides comprehensive routing and level controls for inputs and outputs, along with fast metering.

The only disadvantage of the hardware thru is that you can't hear the signal with your software‑based plug‑in effects, since using these involves taking the long route through the operating system again, and therefore all the inherent delays. But here the soundcard designers have another trick up their sleeves. Many soundcards also come complete with high‑quality, DSP‑based effects on board, and once again your input signal needn't go through the operating system to get to them. As long as the designer has also provided a digital mixer on the card, you can set up its routing so that your input signal can be treated with all the effects you desire, and then sent on to the output socket — all within nanoseconds.

This is one factor that many people don't consider when choosing a soundcard. It may seem better value to buy a soundcard with basic I/O, and then amass a huge array of software plug‑ins, but this can end up costing far more than buying a soundcard with DSP‑based effects already on board. Yes, you could download lots of free software plug‑ins from the Internet, but despite the undoubted talent and enthusiasm of these developers, the best reverb and EQ algorithms take a huge amount of time to develop, and rarely come cheap. In addition, however wonderful they sound, each one will take its share of your processor power.

Many of the latest soundcards are excellent value for money, with some of them offering multichannel recording and playback, digital mixers and multiple effect busses at less than it would have cost to buy a single dedicated mid‑range rackmounting effects unit a few years ago. The real beauty of them is that you don't need a computer with a NASA specification to run them (since most of the hard work is done by the soundcard itself). Best of all, there is virtually zero latency when using any of the effects.

There are now two basic approaches to implementing DSP‑based soundcard effects. Most cards have a fixed set of features and effects. This helps to keep the price down, especially if the same chips have been used on other hardware units. Examples are the Yamaha DSP Factory (with chips also used by the 02R and 03D digital mixers), the SW1000XG (using identical devices as the A3000 sampler and MU100R MIDI module), and the Lexicon Studio (which uses the same core processing engine as the famous PCM90 reverb). However, some of the latest soundcard options provide a user‑defined DSP system, so that you can download your own selection of devices (such as various mixer and effect options) to the generic DSP chips. These then run entirely using DSP power, without putting any extra strain on your main computer processor. If you prefer to use an eclectic mix of effects, this type of system may be more appropriate to your needs. Your choices here include the Creamware Pulsar (and forthcoming Scope system), and the Soundscape Mixtreme.

Driving The Point Home

If your soundcard doesn't have DSP effects or a hardware thru facility then you will have to take the long route through the operating system. However, there are still various ways to minimise latency, and these all involve ways of bypassing some of the slowest bits. The best way to do this is by writing soundcard drivers that communicate directly with the soundcard at a lower level, rather than relying on the operating system to do all the work. Occasionally software developers optimise their products for a single soundcard in exactly this way, particularly in the case of some early software synths. Seer Systems' Reality for the PC was closely linked to the SoundBlaster range of consumer soundcards, and this meant very low latency. Unfortunately, it also limited the market to owners of these cards, and in the first major update the drivers were re‑written to take advantage of Microsoft's DirectX technology. DirectX was primarily written for games, but since it provided much lower level hooks to the operating system, its audio latency was significantly lower. Unfortunately, this is a PC‑only solution, and it only applies to playback, so it's currently not an ideal solution for musicians.

The most successful driver technology available to both Mac and PC owners is Steinberg's ASIO (Audio Stream Input and Output). Initially developed on the Mac to provide the audio streaming engine of the Cubase VST MIDI + Audio sequencer, ASIO also minimises the potential delays by talking directly to the soundcard at a lower level than normally used by the operating system, using specially written driver software. Once again, an ASIO‑specific driver needs to be written for each and every soundcard, but the software interface has been carefully devised so that soundcard manufacturers can develop ASIO drivers for their own soundcards.

Because of the popularity of Cubase VST on both PC and Mac platforms, many soundcard manufacturers have collaborated with Steinberg to write these special drivers, although sometimes company politics have resulted in long delays before they appeared. In addition, Emagic have also collaborated with Steinberg to add ASIO support to the Logic Audio range, which further establishes it as a professional standard. It is a sign of its importance that some professional soundcards such as the Lexicon Studio have initially only shipped with ASIO drivers, on the grounds that this performance boost is far more important than providing Windows Multimedia compatibility. Others are initially shipped with Windows MME drivers for immediate compatibility with a wide range of PC audio software, with ASIO drivers released later for enhanced performance. This is the case with the Event range, the Emu APS, and the Yamaha DSP Factory cards.

Delayed Reaction

The main benefit for those soundcards with ASIO drivers is that latency values are much lower. To give you some idea of the improvements, my Event Gina soundcard has a latency value of 557mS with the recommended settings for its Windows MME drivers. With the newly released ASIO ones this can drop to as low as 25mS. Another example is the Sonorus StudI/O, whose Windows MME drivers have a latency value of 750mS, but whose ASIO drivers have a typical latency of 32mS.

Once latency drops below about 100mS the application transport controls will also start to feel like their hardware equivalents, and the software level meters will move in time with the music you are hearing (see Other Effects Of Latency box). However, 100mS is still not low enough for monitoring a performance — with or without software plug‑in effects. Between 25mS and 50mS, software monitoring starts to sound more like a double tracking effect than an echo. While some musicians might be able to work with it, it will still compromise the performance to some extent. Few soundcards manage latency values below about 25mS, even with ASIO drivers.

Musicians will be most comfortable with a figure of 10mS or less — the equivalent latency you get between pressing a key on a MIDI keyboard and hearing the sound. Those with golden ears claim to be able to hear shifts of 1mS where 'grooves' are concerned.

The reason that the signal can get so delayed is that it has a long way to go.

In Summary

You can now see why the effects of latency are so important to any musician. On‑board digital mixing and DSP‑based effects have transformed many musicians' creative output, but if you want to record any live performances you will only be able to provide suitable headphone or control room monitoring if you have zero latency. If you rely on a soundcard‑based music recording system you really do need hardware thru monitoring or an external mixing desk to combat it.

The alternative for those who leap before they look (rather than the other way round) is to join the ever‑growing groups of people who sit poised on the Internet waiting for new soundcard driver releases with ASIO support. Tweaking software settings to reduce the latency becomes a way of life, and some get so frustrated that they are prepared to trust their systems to Beta (pre‑release) driver versions, simply to achieve the holy grail of low latency. It may be a hard world out there, but at least there are some soft solutions.

Other Effects Of Latency

Apart from leading to difficulties when monitoring, latency causes other problems too. Playback of existing recordings also suffers a similar lag, but in this case the audio recording software can do something about it. The software has to initially fill up its memory buffers before audio playback starts, so it simply waits until these are full before actually starting to scroll the screen display and starting audio replay. Although this means a short wait before anything seems to happen, it does ensure that the audio you hear is glitch‑free and that the waveforms being displayed on screen are completely in sync with it.

However, high latency values also make the transport controls of a software‑based audio recorder seem very sluggish and imprecise — it feels more like making a suggestion than giving a command. This can make punching in and out of existing tracks during overdubs very hit and miss, although the virtual track system adopted by most audio sequencers means that musicians tend to record their overdubs on another track, and then bounce the extra bits down when they have been perfected.

Another side‑effect of latency is that if any software‑based metering of input and output levels is provided, this will also suffer a time delay, making it quite difficult to relate the meter movements to the sounds you are hearing. Although you might think that the software could take account of this, it would mean yet more buffering and, in practice, glitch‑free audio is far more important to developers. However, most of the soundcards that provide hardware monitoring facilities also feature input and output level metering which is far more responsive. This is because, although the soundcard utility is still running in software, it displays the signal levels measured by the soundcard driver, rather than those that emerge from the sequencer buffers, and so much more closely reflects what you are hearing. If you want to monitor levels, therefore, always use the utility provided by the soundcard manufacturer. The main disadvantage is that this will only show the final mixed levels at each input and output, and not the individual instrument sounds before they are mixed together by the sequencer.

Avoiding The Issue

If your main problem with high latency is due to attempts to monitor your input signal during recording, then there is one easy way to avoid the issue altogether, and this is to use an external mixer for monitoring purposes in time honoured fashion. If you want to hear the performance complete with effects (as most vocalists and guitarists do) then use some existing analogue outboard effects for this purpose. However, tap off the signal to be recorded pre‑effects, using a direct channel output or insert point on the mixer, so that the recording itself is made dry. Then you can apply software plug‑in effects afterwards, and still play around with the final effect settings as much as you like. Since the final mix will usually determine the type and level of effects used for each instrument, recording dry is always the safer option anyway.

If you really want to use software plug‑in effects during monitoring, you could try only using a 100 percent wet effect (with reverb for instance), but routing the direct signal using hardware, either using a hardware thru (if available) or an external mixer. In this way, the important direct sound would have zero latency, but you would still get some effect in your headphones as well to help support the sound.

Soundcards With Hardware Thru

In general, any soundcard that features a digital mixer will let you route input signals directly to one of its outputs for zero‑latency monitoring. Here are some that can fulfil this requirement:

  • Creamware Pulsar.
  • Emu APS.
  • Ensoniq PARIS.
  • Event Darla, Gina, Layla.
  • Korg 1212.
  • Lexicon Studio.
  • MOTU 2408.
  • Sonorus STUDI/O.
  • Yamaha DSP Factory and SW1000XG.