Sony SpectraLayers Pro

With some innovative concepts borrowed from photo-manipulation software, Sony aim to make spectral editing creative as well as useful.

Audio layers are shown in the spectral display in green, or in red if selected. The clipboard is blue.

SpectraLayers Pro is a spectral audio editor published by Sony Creative Software but developed by Divide Frame, a small French company who retain its copyright. The program represents audio using a multi-dimensional, spectral display, and allows it to be edited in both the time and the frequency domains.

Spectral audio displays are nothing new, and neither, for that matter, is spectral editing: Izotope's RX and Magix's Sequoia provide a spectral editor, and the concept of using a frequency axis for audio selection crops up in consumer-level programs like Roland's R-Mix. SpectraLayers brings something new to the mix, as it were, in the form of layers, which we'll look at shortly.

Spectral editing is usually presented in the context of audio repair and restoration: individual musical or sonic elements generally occupy distinct areas of the frequency range, so an editor that allows you to cut, copy and paste by frequency as well as time stands the best chance of being able to isolate sound components — in particular, unwanted ones — and remove them from a pre-mixed track without affecting anything else in the mix. A spectral display allows individual sound elements to be identified, selected and manipulated visually, much as image material might be edited and corrected in a package like Photoshop. The kind of processing technology required for spectral editing lends itself well to additional utility features such as noise reduction, which can be thrown in as a bonus: SpectraLayers Pro, like Izotope RX and Sequioa, provides a de-noiser.

While Izotope RX and Sequoia are described as repair and restoration tools, however, SpectraLayers Pro is being presented as more of a creative sound-manipulation program. The difference is, to a large extent, one of marketing: the programs are very similar in their basic functionality and appearance. The distinctive selling point of SpectraLayers Pro is hinted at by its name: it employs the concept of 'layers' to allow non-destructive editing of audio. The term will be familiar to Photoshop users, and provides the same benefits here as there: the ability to organise and audition different edits and alterations without committing to a sequence of irreversible changes.

First Impressions

SpectraLayers officially comes in several versions. The Trial version is a 15-day time-limited free demo, working in mono only at 24kHz, which cannot save projects or export files. SpectraLayers Enterprise is multi-channel and surround sound-aware, but appears not to be shipping yet. The Pro version, reviewed here (and which we'll refer to simply as SpectraLayers), works in stereo at up to 96kHz.

SpectraLayers has a rather hefty hardware requirement: roughly speaking, your computer needs to be no older than three years, and the PC version needs Vista or Windows 7. It was usable on my two-year-old MacBook Pro with 4GB of RAM, but ran a little sluggishly, and made the machine feel very much like minimum-spec hardware. It would, presumably, be even slower alongside a DAW.

The SpectraLayers installer has what could charitably be called a vintage look, but worked fine right up until the licence authentication phase, which timed out on my (slow) broadband line until the third or fourth attempt. SpectraLayers itself would not win any beauty contests: it has been built using the Qt application framework, more suited to Unix than Mac or PC, with quirky, indistinct icons, non-standard menus and crude on-screen controls. While Qt can be used to create nice native applications — AudioMulch being an example — SpectraLayers clearly needs more effort.

The bundled documentation is, similarly, rather weak: a set of help pages is included which can be read in a Web browser, but the material is terse and incomplete, and it sports a few typographical errors.

The shortcomings of the application design and the documentation may well be due to its early release as a Sony product (we reviewed version 1.0.18), and I'm sure things will improve as new releases come along, but the current version did leave me scratching my head on numerous occasions, wondering why certain buttons were changing colour and icons were taking on an appearance not documented or explained anywhere. I'll attempt to fill in the important gaps during this review.

Screen Layout

SpectraLayers has an overall appearance like that of most audio editors, with a large, central audio-display panel surrounded by icons and editing panels. Along the top toolbar are icons for the main selection, editing, viewing and playback functions, while the left-hand margin contains buttons for specific editing modes. The right-hand side features sub-panels for undo/redo history, audio channels and layers. These panels can be resized, rearranged, closed, and even dragged to the left side of the audio panel, but unfortunately don't operate as floating windows.

The central display area is a spectral graph of what SpectraLayers calls the 'mix', which includes all layers currently enabled for playback. The audio channels of the mix — of which there are two if you're working on a stereo recording — are stacked vertically in panes, with time represented on the horizontal axis and frequency vertically. Colour intensity indicates amplitude. (For a newly imported file, the display will be green; I'll describe the multi-colour highlighting shortly.)

The display can be scaled or scrolled vertically, to focus it on particular frequency ranges, or horizontally to home in on a particular section of the recording. The intensity and contrast of the colour highlighting can also be varied, although contrast is described as 'gamma correction' — complete with a Greek letter as icon — which is a little obscure. A '3D displace' control rather cleverly projects the amplitude dimension at your choice of diagonal angle, in what my technical drawing teacher would have described as oblique projection.Amplitude can be shown on a diagonal axis for a 3D look.

Layers

All editing in SpectraLayers is performed on layers. A layer might correspond to a specific audio file, or to a portion of audio that has been selected and clipped from a file. While a conventional audio editor might perform edits directly, backed up with undo and redo operations for when things go wrong, SpectraLayers allows edits on a file to affect a separate layer, leaving the original audio unaltered. Layers can be created, and deleted, whenever you wish. Once an edit has been performed, it's easy to do an A/B test to hear the result: mute or unmute the layer that was targeted by the edit. You can do multiple, alternate, edits into their own layers and decide later which, if any, to keep.

Every layer has a button to invert its playback polarity, so one obvious way to work with SpectraLayers is to isolate an audio artifact within a file, copy it into its own layer, and then add that layer to the mix with its polarity inverted, so that it cancels the signal in the original. Each layer also has a playback-level fader, so that the effect of an edit can be boosted or attenuated.

A new, empty layer can be created at any time via the New Layer command in the Layer menu or the Layers pane's contextual menu. To open an audio file for editing, it must be read into its own layer via the Import Layer command.

The Layers pane itself is a little confusing, since it resembles a simple multitrack mixer but is also used for editing operations. Each layer has a level fader (labelled in dB) and a play/mute button indicated by a loudspeaker icon. Play-enabled layers are shown in green with a loudspeaker, muted ones in grey with the speaker absent. A muted layer is taken out of the spectral display completely, while the level setting affects the brightness with which a layer appears in the display: in other words, the Layers pane is creating a mix which is reflected in the spectral pane.

Now things get a little confusing: click on a layer in the Layers pane, and it turns red, as does its spectral information in the main panel. This layer is now the target for editing operations, and in particular, for pasting selected audio. What the manual does not explain is that the target layer is also soloed for playback: its speaker icon retains little graphical sound waves, which are removed from the other layers. For added confusion, a layer may be muted but also a target: it is still soloed, so gets labelled with speaker sound waves but no speaker! This does all make sense, but I spent about an hour trying to work out what the bits of speaker icon were telling me, and I'm sure there must be better ways of indicating what's going on. There is one nasty gotcha here: right-click on a layer and select Delete to (permanently) delete it, and the red, targeted layer is the one that is deleted — not the one clicked. As a piece of interface design, this is misconceived, to say the least.

Editing

Audio selections within layers are made by clicking the Select icon in the toolbar, and the selected area is shown as a 'marquee' with a line around it. Like many of the toolbar icons, Select actually offers several functions, and you need to hold down the mouse button to see them all. Selection can be by rectangular area, or purely by time (encompassing the whole frequency range), or purely by frequency (encompassing the whole time range). The toolbar also provides Move and Scale tools, which you might expect would shift or transpose the selected audio — certainly, Sony's web site claims this functionality — but in practice, it's only the selection marquee that moves.

Once selected, a chunk of audio can be copied to the clipboard, the contents of which are superimposed on the spectral display in blue. The clipboard is rather fragile, though: click somewhere else in the display and it is thrown away. Since SpectraLayers is big on non-destructive editing, you are encouraged to copy audio into dedicated layers before altering it. In general, non-destructive operations like Copy can be performed on layers in the mix, but a target layer has to be selected before destructive operations like Cut, or effects processing, are allowed.

You would expect to be able to shift layers around in time; for example, to extract sonic events from a mix and move them without affecting the rest of the mix. Sure enough, the Edit menu provides Insert and Delete Time functions, but you have to be careful not to shift sections of the entire mix rather than the desired layer, and the operations can't be undone, which is a criminal omission. Insert and Delete Time also resulted in a fair number of application crashes in the release I was testing.SpectraLayers' extraction tools let you 'paint' a region of the spectrum onto a separate layer, for further editing.

There are also 'extraction' tools, which work directly between layers, without using the clipboard: when you 'paint' in the spectral display, audio is copied from the mix directly into the selected layer. Extract/Area takes whatever is under the cursor, with adjustable accuracy and 'blur' in the frequency and time dimensions. Extract/Frequency and Extract/Harmonics are much more interesting: SpectraLayers attempts to track the dominant frequency, or frequency and harmonics respectively, of the audio under the cursor, and paste this across. In the 'Harmonics' screen, I've manually traced one of the low harmonics of a rather complex bell tone; SpectraLayers has highlighted this frequency and its natural harmonics, which are copied into their own layer. Extraction only works in an area which has already been selected for editing — this caused a bit of head-scratching — and there are tolerance and time-limit figures which can be tuned to improve the selection.

Harmonics: when working with tonal material, SpectraLayers can pick out harmonics (top) and transfer them to a layer of their own (bottom).Noise reduction proved to be a slightly confusing affair, mainly because SpectraLayers offers a lot of flexibility but rather less in the way of documentation. A noise-reduction pass involves registering an area of noise first, as a reference — so far, so conventional — but the selection is done in two dimensions with a paint tool, and then the actual reduction is also performed by painting, so it's not totally clear how the two selections interact. In practice, and after a few attempts, the system does seem to work: the registration does indeed create a noise signature for the selected region, at least in the frequency domain, and the program helpfully highlights the affected area in blue when applying the signature to a noisy passage. Since the editing is done in layers, it's easy to do multiple, alternate, edits and audition them separately — or together. Noise reduction is susceptible to artifacts, as always, and needs practice; unfortunately, without decent documentation it's difficult to know exactly what's going on.

In general, the way in which the extraction tools function is a little irritating: it's not possible to listen to the audio and then immediately extract it, since extraction needs a selected target layer, and when a target layer is selected, it is soloed, muting whatever you're trying to edit! This is the kind of design glitch that I'd expect to see cleaned up in an incremental release.

The Modify tools, which operate destructively on the selected layer, are, again, suggestive of a painting program: Amplify and Eraser do what you'd expect (but, again, only operate within the selected region), while Clone appears to be a more complicated affair. I was expecting the same behaviour as a paint program, where you anchor the start point for sampling and then paint into the destination, with the source point tracking the destination in parallel, but after a fair amount of mousing around, I didn't manage to get an outcome that suggested this was happening. Often, I would get what looked like multiple copies of the original strewn across the spectral landscape. Without decent documentation, it's hard to say whether this was pilot error or malfunctioning software.

Further Processing

Here, I'm applying Audio Damage's Kombinat Dva to a narrow frequency band. The results are visible in the 3D view of the recombined mix.

As well as its own editing tools, SpectraLayers supports 64-bit VST plug-ins natively, and 32-bit ones via a compatibility layer. Effects are rendered destructively on the selected region of the spectral display in a selected layer. Both 32-bit and 64-bit effects seemed to work, and although an effect can be previewed, this takes place in isolation, not alongside other frequencies in the same time region. Happily, the undo function works for effects.

Putting VST effects on just part of the frequency spectrum of an audio file seemed like a contender for some creative high jinks, so I took a drum loop from a sample library, extracted a couple of frequency bands into their own layers, and then applied two presets from my current favourite effect, Audio Damage's Kombinat Dva, to the bands. The screen nearby shows one band: the original audio is displayed in red, while the effected audio is in blue (the clipboard colour). This spectral view shows clearly how much additional harmonic material is added to the original band by the effect. In the next screen, I've added an effected layer back into the entire mix, keeping it highlighted in red (and I've also switched to the 3D view). A quick glance at the Layers pane shows that I've effected a copy of the frequency band layer: the original layer is still present and has been phase-inverted to take the 'dry' version out of the mix.

There's not a lot here that I couldn't have done, after a fashion, with a conventional mixer and a bunch of EQ plug-ins, but SpectraLayers allows much more detailed and tighter control of the selected material, the layers allow for easy auditioning, and the visuals show exactly what's going on.

Conclusions

This is an early release of SpectraLayers under the Sony brand, so it's probably fair to regard it as a beta release from a small company with limited resources as it makes its way into the wider world. The current version has some serious faults: it's rather unstable, the user interface is cranky, to say the least, features are missing, many important editing tasks are rather too difficult to perform reliably, and the documentation is woefully inadequate. On the other hand, the basic approach whereby the user works non-destructively, extracting audio components and artifacts into separate layers, processes and recombines them, is an engaging and potentially powerful process, and this way of working lends itself to all kinds of creative sound design tasks in addition to restoration and clean-up. Hopefully, with the resources of Sony to back it up, SpectraLayers will quickly turn into a world-class sound editing environment.