Backup Strategies for the PC Musician

I know — backups are boring. But just imagine for a moment how you'd feel if your PC's hard drive broke down, or (even worse) that your PC was stolen or reduced to ashes in a house fire. Most of the contents of your Windows partition could be re-installed onto a new hard drive and, assuming you've got your gear properly insured, the insurance company would replace your PC in the case of fire or theft, but your personal data is quite another matter.

Your accumulation of letters, accounts, songs, associated audio files, plug-in presets and the like may have taken weeks, months, or even years to create. One of the aforementioned disasters could wipe out all this effort and inspiration in a few moments, with no way of getting it back — unless you've made some backups, that is!

So, after that stomach-churning scenario, there's a simple question you can ask yourself to determine how often you should be taking backups: how much work am I prepared to lose? For some hobbyists backing up once a month may be enough, if they only spend a few evenings and the odd weekend making music, but for most of us once a week is more appropriate, and after an important session, straight away.

But where's the best place to store your backups? What's the easiest way to make them, and what should they include? Can you use compression to make them smaller without losing any data? If the worst happens and you suffer from a bad crash before making backups, it possible to recover data from a trashed hard drive? Let's see.

Most 'high-street' PCs still leave their hard drive as a single vast storage device for Windows, its applications, and your data, but I hope most SOS readers will by now recognise the benefits of keeping their personal data elsewhere, on a different partition or drive.

For a start, this can make backing up a lot easier. My SOS articles are all on one partition, for instance, and my audio projects on another, my Giga-format sample libraries on another, and so on, which means that I can back up each entire partition separately without having to pick and choose particular files and folders. Let's face it, the easier you make the backup process, the more likely you are to do it, and the safer your data will become.

Organising your data into different partitions will also make finding it somewhat easier in the future, since — having taken the trouble to come up with a scheme to divide it into suitable categories — you'll always know the best place to store future files of each type. I know exactly where all the latest update files for my various plug-ins and music applications are, since I've created a separate 'Updates' partition containing folders named 'MusicUpdates' and 'PCUpdates', each containing further sub-folders divided into categories such as Plug-ins, Utilities, and so on, within which there are further folders arranged by the developer's name, plus a master text file containing every password and set of registration details I've received to date, in alphabetical order.

While some people might find this level of organisation amusing, the fact is that even in the event of a calamity that takes out my Windows partition or causes one of my hard drives to fail, I can start again by reformatting my Windows partition, or even buy a new drive, safe in the knowledge that as soon as I've reinstalled my applications, a backup of my Updates partition will still contain all the files I need to bring my applications back to their newest versions, as well as letting me re-register the majority of them without having to convince the developers that I'm not trying to pull a fast one.

If you have several Windows partitions, either to run different versions of Windows (I still run a Windows 98SE one for various elderly applications and hardware without Windows XP drivers) or several instances of the same Windows version (XP for general applications, and a stripped-down XP version solely for music applications, say), separating your personal data out into several partitions also means you can access it from any Windows installation. It's also easier to keep each one tidy and defragmented for faster performance.

Some musicians even install their applications onto a separate partition from Windows, but I've yet to see any advantage in this approach. Yes, it does let you create a smaller Windows partition, but if you ever have to reformat that, or restore an earlier version, you'll still have to reinstall the applications to update the Windows Registry so that it's aware of them.

Occasionally, making sure your personal data is on a different partition from the associated application can be difficult. Cubase, for instance, defaults to storing plug-in and soft synth presets within its 'vstplugins' folder, and while this is adequate on a day-to-day basis, I much prefer to navigate to my global folder, named 'Software Presets', on a different data-only partition, in which I've systematically stored the bank of presets supplied by the manufacturer, plus every preset I've ever created on any of my various music partitions over the years, for every plug-in and soft synth I own.

This partition is only 14.2MB in size, but it currently contains 581 vital files that encompass a huge amount of personal effort, but is easily backed up or transported. I recently bought a new laptop, and was able to copy this folder and all its contents across to it within seconds. Keeping a separate partition also enables me to access presets from several different Windows versions if I ever need to.

File-naming Tips

By adopting a numeric approach to the naming of songs, you can easily associate various different file types with each song version you save (left-hand Explorer window), while when you open a project in your sequencer they can easily be sorted into chronological order (right-hand Open Project window).

Adopt an informative naming scheme for your songs. You don't need a perfect name for each, but a descriptive title like 'floaty guitar' or 'weird drums' will make more sense than 'newidea' or 'demo 47' six months from now.

Make the final two characters of your project name a version number — so the first version you save is (for example) 'dreamtime01'. Each time you make a significant change, save the new version with an incremental change to the number, as in dreamtime02, and so on. This not only acts as a long-term undo function, in case you change your mind about the direction of the song, but also ensures that when you load a song version into your sequencer you can view the versions in chronological order.

Adopt the same file-name convention for associated files if they change during the course of the song's creation, so you know which one belongs to which version of the song. This is generally made easier because the filename's extender is different in each case, so you can use the same body name for each one. For instance, I've got a folder named Moonhill that contains files named Moonhill 15.cpr (Cubase SX project), Moonhill 15.gsp (GigaStudio project), and Moonhill 15.xpa (for XGpad), which all appear in Explorer with the appropriate icons to associate them with their application. Even if you want to return to an earlier version of a song, you can then back-track with everything in sync.

If your sequencer allows automatic backups at certain time intervals, these will generally be given a slightly modified file-name, or just a different extender, so you can still see which backup relates to which version of the song. When the song is completely finished I generally delete all but the two most recent versions of each song, to save backup space, safe in the knowledge that I've got two complete sets of data to accompany my single set of audio files.

What To Include In Backups

While letters and accounts are one-off files, which makes backup easy, song data is often more scattered. MIDI-only projects will probably contain everything in just one file, but there's no point in religiously saving song project files without the audio data associated with them. Most MIDI + Audio sequencers now encourage you to create a separate audio folder within the song folder, to keep such data together, or even do it for you automatically, but it's vital to save other data relevant to that song in there as well.

This includes such things as hardware synth presets (my songs often use a bank of specially-created SW1000XG presets, for instance), the song data belonging to any other stand-alone applications such as GigaStudio, and of course any user-created presets for effect plug-ins and soft synths. Hands up all those who have decided to reformat their hard drives, safe in the knowledge that their song data was safe elsewhere, only to discover when next trying to load them that a specially tweaked soft synth sound had gone for ever?

Samples are another class of song-related data, but here you must decide for yourself on a backup policy. Personally, I only save sample data with a song if it's been specially recorded or modified for that song. The vast majority of the time my sample data comes from libraries that I still have, both on the original CD-ROMs, and on my hard drive, so I can always recreate my songs in years to come if I need to.

On the other hand, if you're using samples or sample mappings unique to a particular song, or need to re-create that song at a friend's house or commercial studio, it's vital to include them in the backup. This also applies to effect plug-ins and soft synths — if you are satisfied that each audio track sounds as good as it can be, 'freeze' it with the effects embedded (see 'Sharing Data' for more details).

You may find that your particular MIDI + Audio sequencer application provides special functions to help you organise the files that are needed by a particular song. For instance, Cubase SX has a 'Prepare Archive' function in the audio Pool menu that copies any files required by the current project into its folder, and asks whether or not you want to Freeze Edits if any processing has been applied. Once files have been prepared in this way you can safely archive the entire project folder.

Data Recovery

If the worst happens — you lose some of your precious data and don't have a backup — is there anything you can do? Well, it all depends on how the data was lost, and what tools you have available. If you've simply deleted the files accidentally, it's important to try to rescue them as quickly as possible after the event. As I explained in PC Notes January 2004, although deleting files doesn't trash the data, it does mark the space it occupies as unused, and Windows will then start to overwrite it with new data.

If you've already got a suitable utility, such as File Rescue Plus (www.softwareshelf.co.uk), or one of the freeware offerings you can find on the web (try www.snapfiles.com/Freeware/system/fwdatarecovery.html), run it straight away, but remember to save any data files you retrieve onto a different partition or drive, to prevent them overwriting the remainder of your lost data.

Recovery utilities work by scanning for valid files marked as deleted, but with their internal links still trackable. If a particular file that you're desperate to retrieve doesn't show up in the list of recoverable files you may still be able to find some or all of the data by hand with a suitable lower-level utility like the DOS-based 'DiskEdit' of Norton's Utilities suite. However, this can be tedious work, even for an expert, and you'll have to search for specific strings of characters to make any headway. If your hard drive isn't well defragmented, the files may also be in several fragments spread all over the drive, further complicating the process. I've used DiskEdit to rescue various text-based articles after a bad crash, but it's unlikely that you'll track down song data or audio files.

By the way, if you're selling your PC the fact that sensitive data doesn't get wiped from your drives can also be a cause for concern. The answer to to use a Secure File Deletion utility like the freeware Ultrawipe (www.snapfiles.com/get/ultrawipe.html) to write innocuous data into the free space, or use similar options in utility suites

Different Media

Organising your data onto various different partitions does make it less prone to disasters if your Windows partition ever gets corrupted, but it doesn't count as a backup — a true backup is a second or further copy of the same data in a totally different location that you can use to retrieve one or more files if the originals ever get corrupted, accidentally deleted, or otherwise trashed.

Backups should obviously be stored away from the original data to be safe — the further away the better, in my opinion. The quickest and easiest destination is another partition on the same drive, but this won't save you if the drive ever fails, so for the vast majority of musicians with two hard drives (one for Windows and its applications, and the other for Audio and other data) the easiest safe option is to store backups on a partition on the other drive.

Audio data should therefore be backed up onto a new partition on your Windows drive, while data stored in its own partition on the Windows drive should be backed up to a backup partition on the Audio drive. Always use the innermost partitions of each drive for backups, since these will have the slowest performance, reserving the fastest outer ones for audio.

The above is a reasonably safe backup option, but a much safer one is to back up to a completely separate medium — either an external Firewire or USB hard drive, a standard IDE hard drive in a caddy (see SOS June 2004 for more details on these), or onto an optical disk format such as a CD-R, CD-R/W or DVD-R. If you've got more than one PC you can also network them together and back up data from one to the other. Nowadays you don't even need to connect them together using cables — with wireless technology you could back up data from your flashy new music PC to an old 486 model in the garage, shed, or attic!

Hard drives are useful for backing up audio project data because of their high transfer rates (dragging a bunch of audio files and dropping them onto another hard drive takes far less time than burning a CD-R), and they are particularly handy in commercial studios for storing fairly 'active' projects where the band may return and want to do some more overdubs or mixdowns, since you can plug in the drive and carry on using its contents without going through a restore process.

However, once you've deleted the original on your internal drive, bear in mind that the external hard drive copy becomes the master, so a second safety backup will need to be made for security purposes. Moreover, because hard drives are mechanical devices they are more prone to accidental damage than a CD-R or DVD-R disk, so I don't consider them suitable for long-term archiving.

There's still a lot of confusion out there about the different DVD formats, which is hardly surprising, considering the battles that are still raging between manufacturers, and the fact that there are six recordable versions of DVD. DVD-R and DVD+R can record data once, just like CD-R, whereas DVD-RAM, DVD-RW, and DVD+RW can be rewritten loads of times, like CD-R/W disks.

DVD-R is probably the best format for one-off backups, as it's compatible with most DVD drives and players, while Pioneer's rewritable DVD-RW is also widely compatible. DVD+R and DVD+RW formats are only slightly different from these, and were introduced by Philips and Sony among others, but are not supported by the official DVD Forum, although in practice this doesn't really make much difference.

The safest thing is to make sure you buy a dual-format 'combi' DVD recorder that is compatible with both camps, and then you should be able to read and write to any disc you receive. Many drives simultaneously support the burning of CD-R/W discs, making a complete solution that has already resulted in some musicians simply fitting a single DVD recorder that addresses all their burning needs.

All the DVD formats have the huge advantage that they can store up to 4.7GB of data, compared with just 650MB for a CD-ROM, which makes far more sense in these days of huge 24-bit/96kHz audio files. However, DVD-RAM can be even more attractive to some because it offers drag-and-drop functionality like a hard drive, although it's not compatible with as many drives. In its favour, it does support double-sided as well as single-sided media, which extends its storage capability to a massive 9.4GB but, as you might expect, the blank media is considerably more expensive.

While authoring applications for DVD-Audio are still expensive, for writing data it's a completely different story, with even Ahead's budget Nero package supporting DVD data writing. Many computer-based DVD recorders are even bundled with Nero Express, which supports both CD and DVD writing, so you probably won't have to buy any software for backups.

However, although it sounds great in theory to be able to treat a CD-R/W or DVD-RAM disc like a hard drive, saving, loading and erasing files whenever you like, there are various incompatible packet-writing formats involved, and you may not be able to read such backups if you change your writing software. I abandoned CD-R/W packet writing years ago for this reason, and have been quite happy using the Multisession data CD format to make incremental backups (as you can with DVD+R or DVD-R). There's also a school of thought that sees a one-off CD-R, DVD+R or DVD-R format as far more suitable for backup purposes than an erasable one. By the way, for the longest life, do keep your CD and DVD backups out of direct sunlight.

File Compression

If you don't already have a dedicated audio compression utility built into your audio applications, the freeware Monkey's Audio is a perfect way to squash your audio data way smaller than a zip file, as you can see here.

When you're making backups, there's no point in wasting space by storing files at exactly the same size as the originals when there are free and easy-to-use compression algorithms that can squash them to 50 percent or less. I'm not talking about lossy formats like MP3, that discard information not likely to be heard — unsuitable for audio archiving, and useless for data.

For PC users, the most common lossless (bit-for-bit copy) compressed file format is ZIP, which is supported directly under Windows XP, and can easily be added to other versions of Windows by downloading WinZip (www.winzip.com) or a similar utility. However, while ZIP files are good for data, they rarely make much of a dent in audio file sizes. For example, a typical 650MB CD Audio album might only compress down as far as 590MB. RAR format (as supported by Winrar, among others) is significantly better, and might squash the same album down to around 400MB, but the very best I've come across is the freeware Monkey's Audio (www.monkeysaudio.com), which is optimised for lossless audio compression, and is likely to squash the same album down to about 350MB.

Currently up to version 3.99, the Monkey's Audio compressor is not only suitable for backup purposes, but also has a file format (called APE) that is directly supported by various ripper and player utilities such as Winamp, Media Center, and the highly regarded Exact Audio Copy. It has built-in error detection to ensure that any non-perfect decompression is reported, and runs on Windows 9x, NT, 2000, and XP. Similar compression possibilities are available to Wavelab users via that program's lossless OSQ (Original Sound Quality) algorithm.

Dedicated Software

While in many cases you can just drag a bunch of files across from source to destination to make a backup, I've never found Windows Explorer totally reliable when copying a vast nested folder structure, and it sometimes bombs out partway through if it finds a path-name that it considers too long.

For routine backup of assorted file types, 'zipping' them into a single compressed file that retains the nested folder structure when expanded is probably the easiest option (but see 'File Compression' for more related options). You can view the contents of the zipped file at any time, extract them individually or en masse, and delete files within the backup whenever you like.

However, if you want to implement a more intelligent backup regime it's worth investigating the various dedicated software backup applications, which may make your life a lot easier (or not, as the case may be). Some years ago I used to use Microsoft's Backup application (bundled free with Windows), which lets you create a Backup Job containing any combination of files and folders, verify its contents, and Restore any of all of them, either to the original location or a new one. It also provided the option to back up only new and changed files, rather than blindly copying the lot every time, and to ignore specified file types (so, for instance, you could copy your latest audio files, while ignoring any with the BAK extension), as well as offering file compression and password protection. It created a single large backup file, which in my experience is easier to deal with than a huge number of individual ones, although the format used was a proprietary one, which isn't quite as convenient as a ZIP file that you could open in a decade when the original backup application has disappeared without trace.

A dedicated app such as Backup can work well for lots of musicians (and you may find that your particular sequencer package offers its own version), but I rapidly found that I was re-organising my hard drive too often for it to cope, as the original destination folder hierarchy had changed, so I had to create a new job nearly every time I used it, which made life more complicated.

A far more flexible approach is adopted by Wavelab, with its Backup Plan, which is a comprehensive way to save any project data created by Wavelab or any other application. As you might expect, you can choose specific files and folders to include and exclude, but it's the filtering system that raises it to new heights. You can either specify a set of file-types to include (the best option if you're really organised) or a set to exclude (an easier option if you want to include every file except backups, waveform overviews and the like, and don't want to run the risk of missing the odd different file type), you can view the final list of files that gets through your filters in the Preview page, to perfect your Pattern of filters, and can then save these Pattern lists as a basis for future project backups.

Once you're organised, creating a new backup is just a matter of pointing to the project folder, clicking on the 'Include All Files' and 'Also applies to sub-folders' tick boxes, loading your filter Pattern, and then saving the resulting set of files en masse, in ZIP format, to a hard drive, or split across one or more CD-Rs or DVDs. There's also the option of compressing any audio files using Wavelab's proprietary OSQ format (see 'File Compression' box).

If your data partitions are small enough and sufficiently organised, you can even use imaging software such as Norton's Ghost and Powerquest's Drive Image to save them in their entirety, just as you may already do with your Windows partitions. This is an ideal approach for general-purpose data, updates, and so on, where you want to be sure you back up absolutely everything in the partition. These applications also provide compression and selective extraction options.

Whether you're using external hard drives, CD-R or DVD-R discs for backup, for maximum security it's important to store them well away from your PC. Data corruption is easy to recover from if you've got a backup on CD sitting next to the computer, but it isn't if you have a fire and both get trashed. At the very least, you should store your backups elsewhere in the house, and preferably at a completely different location — many friends collaborate and store each other's backups, or if you have a day job you could store your music backups at your workplace. Don't trust any backup regime without occasionally trying to restore your project data into a different location and running it from there — this may pick up missing files that you've accidentally omitted from the backup.

Once you're armed with these techniques, your data should be safe from attack by hardware failure, fire, theft, and most other calamities. You might sleep more soundly at night as well!