You are here

Uniting Music & Film

Not content with reviving and staging a 1920s avant-garde work for the first time, Paul D. Lehrman also worked out a way to reunite the piece with its contemporary film, which was never used due to technical problems. This is how...

A couple of years ago I wrote a two-part article for Sound On Sound about my work to revive and bring about the first performance of George Antheil's glorious celebration of cacophony, the Ballet Mécanique, in its original 1924 orchestration: three xylophones, four bass drums, a tam-tam, two pianos, a siren, seven electric bells, three airplane propellors, and 16 synchronised player pianos playing four different parts. It might be worth re-reading that article before you go ahead with this one, to hear about the long and sometimes painful history of this piece, which has been called "one of the most important lost works of the 20th century". The article is in SOS August 2000 and September 2000. For a quick summary, read the box about Antheil and the Ballet Mécanique opposite.

Ballet Mécanique As A Film

MOVING PICTURESOne important aspect of the piece that I didn't discuss in the original SOS article was that this wild piece was in fact originally conceived as a film score. The French Dadaist painter Fernand Léger, photographer Man Ray, and an American cinematographer named Dudley Murphy created a film, also called Ballet Mécanique, for which Antheil's music was supposed to be the accompaniment. The film, which runs about 16 minutes, is a highly abstract collage of familiar and unfamiliar images (some of which you can see in the montage above): a man's head in a kaleidoscope, a woman on a swing, half-a-dozen kitchen funnels dancing on a string, an animated Cubist caricature of Charlie Chaplin, pistons, dynamos, roulette wheels, carnival rides, and dozens of other images, cut together in the Dadaist fashion — meaning almost at random — repeating and looping, backwards, forwards, and upside down. In other words, except for the fact that it's in black and white, it looks very much like a music video, about 60 years before the form was invented!

It's impossible to say whether the music or the film came first, since there are reports favouring both sides. What is known is that the two never worked together as planned. The technology for putting an optical soundtrack onto a film hadn't been invented yet, but somehow Antheil and the filmmakers thought they could combine and synchronise their work! Unfortunately, it appears they prepared their parts of the project in isolation from each other, and the first time Antheil presented his music to the filmmakers (or vice versa), it was discovered that there was twice as much music as there was film. Consequently, Antheil had his Paris premieres without the film, while the filmmakers premiered their work in Vienna, at an Exhibition organised by architect and sculptor Frederick Kiesler, without music.

Over time, there have been several attempts to present the film with Antheil's music, including a performance using only a single player piano (and no human performers) given by the composer in 1935, at the Museum of Modern Art in New York City. But until just last year, no one has tried to match Antheil's original concept for the score with the film.

About George Antheil & Ballet Mécanique

George Antheil.George Antheil.George Antheil, a young American pianist and composer living in Paris, wrote what was to become his most famous work, the Ballet Mécanique, in 1924, for multiple electromechanical player pianos. The piece was never performed as he originally envisioned it, because synchronising multiple player pianos was simply impossible — but he didn't know that when he started. When he found out his piece was unplayable, he reconfigured it for a single player piano and multiple human pianists. This version was performed in Paris in 1926, where it was applauded for its scandalous nature, and the following year in New York at Carnegie Hall, where it unfortunately was received very badly — so badly, in fact, that Antheil's reputation as a composer never recovered. It was revived for one performance in 1989, but Antheil's original 1924 vision had to wait a little longer. In the mid-'90s, G Schirmer, the music publisher who had taken over Antheil's catalogue, came up with the idea of using MIDI-controlled player pianos to realise Antheil's original conception.

Schirmer hired me to do the programming, which involved entering some 400,000 notes and over 600 time-signature changes into a sequencer. But I did more than that: I gathered aeroplane and siren samples, built a MIDI-driven 'bell box', and, with the help of Yamaha (who loaned us 16 of their Disklavier MIDI-controlled acoustic player pianos), I arranged for the first performance of the piece at the University of Massachusetts Lowell, where I was on the faculty at the time. We also made a recording of the piece, and that recording (along with other works for percussion and multiple player pianos) is now available on a CD on the EMF label ( Since the premiere, the piece has been performed in New York (at Carnegie Hall), San Francisco, and in Canada, Belgium, and Holland.

For more information on this project, and the touring schedule for the film, visit

A New Old Film

Like many early films, there are a number of versions of the Ballet Mécanique floating around, each one claiming more or less to be 'authentic', with scenes in different order or even missing entirely. However, in 1975, a print of the film was discovered which has a legitimate right to be considered definitive. In November of that year, Frederick Kiesler's widow, Lillian, found an ancient can of hand-spliced film in the closet of their weekend home in Germany. On the can was a label: 'Leger'. The cans found their way to Jonas Mekas, a filmmaker and scholar, and director of Anthology Film Archives in New York, who realised that this was Léger's personal print of the film, the one which Kiesler most likely had presented at the world premiere. The image quality was, in Mekas's words, "fantastic".

Last year, Anthology Film Archives began a world-wide crusade to bring the film to a wider audience. Ballet Mécanique is one of some 160 avant-garde films, made from 1894 to 1941, in a highly ambitious travelling exhibition entitled Unseen Cinema. The program is slated to be shown in 25 different venues around the world over the next five years. The first showings were in June, 2001, at the Moscow International Film Festival and in September, in New York (a visit is planned to the UK next year — see the 'Sound & Vision' box later in this article).

Bruce Posner, a film archivist working with Anthology Film Archives, had heard about my revival of Antheil's original score, and contacted me, wondering if I would be interested in adapting the music to fit this exhibition print. In order to convince me, he sent a VHS copy of the film, onto which he had crudely laid some of the music from my CD, by simply putting a mic next to a speaker. The correspondence between the sound and images was fascinating, even though, sure enough, only half of the piece had played by the time the film was over.

There would be no monetary compensation for the collaboration, Posner explained, at least not at first, but it would be a chance to finish an unfinished work — like the score itself — from 75 years ago. Just as I did when I was first asked three years ago to get involved with the Ballet Mécanique, I jumped at the chance.

Film Speed & Music Tempo

There were many decisions to be made and obstacles to overcome. The first decision concerned the projection speed of the film. Before motion pictures started to talk in the late 1920s, there was no standard for how many frames of film were supposed to go through a projector each second. Frame rates as low as 16fps were used, which is one of the reasons why old silent movies today often look so comically speeded up: we project them at 24fps, which is now the standard, but it's not what their makers intended.

Posner viewed his copy of the film at several speeds, and decided that the action on the screen looked the most natural at 20fps. He had the film transferred to Beta SP videotape at that speed, and then sent me a VHS copy of the tape. I immediately made a dub of the tape, striping it with SMPTE timecode and window burn, so I could sync my sequencer and audio-editing programs to it. I then did what I always do when I start to score a film: I went through it and marked down all the possible hit points — mostly changes of scene, of which there were dozens — where I might want to have a musical accent or transition. I entered these SMPTE numbers into my sequencer: Opcode's late, lamented Studio Vision.

Yamaha's Disklavier, the MIDI-capable digital piano used for the live performances of Ballet Mécanique.Yamaha's Disklavier, the MIDI-capable digital piano used for the live performances of Ballet Mécanique.Now I had the flexibility to change the tempo of the music wherever I wanted, so that I could make all the hit points, right? Wrong! Antheil's score doesn't have any tempo changes in it, so adding a lot of changes, even subtle ones, would be getting very far away from his original concept. But an even more important factor was that I had this glorious recording of the Ballet Mécanique with 16 Yamaha Disklavier acoustic player pianos already on tape, which I wanted very much to use for the film score. Changing tempos in that recording, while certainly possible with DSP tools, would not be easy, and might sound very strange.

George Antheil specified a tempo of about 150bpm, but the piece is clearly unplayable at that speed, either by man or machine. The Lowell premiere (and my recording CD) was at 100bpm. Since then, there have been performances at 115bpm, and that's about as fast as humans can be expected to play it. The Disklaviers have their own limits, too: since they have a true physical piano action, there is a minimum recovery time for all that wood and metal to come back to its starting point. As I was preparing the sequence files, I tested them on a Disklavier, and was able to get the speed up to 133bpm before the instruments started missing rapidly repeating notes and chords.

Fortunately, right after the recording session for the CD, while we still had all those Disklaviers on the stage at Lowell, I did another recording, of just the Disklaviers and the electric bells, at the pianos' maximum speed: 133bpm. Why? Because I could! I didn't know what I might use the recording for, but I felt it would be a good thing to have in the can.

Like the other recordings, this machines-only take was done on 24 tracks, using three Tascam DA88s. The sequencer playing the pianos was locked to SMPTE timecode and the tape decks were locked to word clock, both of which were coming from a Mark Of The Unicorn Digital Time Piece, which in turn was locked to a digital video camera that was taping the proceedings. Having a common clock source for all of the devices was a crucial factor — just how crucial, I wouldn't know until later on.

At the original tempo of 133bpm, the audio and MIDI tracks did not match at the end of the unedited 25-minute piece.At the original tempo of 133bpm, the audio and MIDI tracks did not match at the end of the unedited 25-minute piece.

I had another trick up my sleeve: early on in the project, when I was creating the sequences of the player-piano parts, I took an extra day and also sequenced all the live percussion parts — the xylophones, bass drums, and tam-tam — as well as the two human-played piano parts. This was strictly for my own benefit, so I could hear what the piece sounded like before we started rehearsing it. I still had those files, which of course, being MIDI sequences, could be played at any speed. When Posner approached me with the idea of scoring the film, I realised I could combine those MIDI tracks with the audio player-piano-only tracks, and create a hybrid MIDI/acoustic performance at 133bpm, which would be my film score.

So I tried it. I transferred the player-piano tracks into Pro Tools (it turned out I only needed eight of the 24: a hypercardioid stereo microphone pair, a spaced omni pair, and single mics on each of the four groups of Disklaviers), and then took the individual audio files and pasted them into Studio Vision. The instrumental MIDI tracks were laid on top. The bass drums and tam-tam I played from an Emu Proteus 1, the xylophones on a Kurzweil K2500, and the live pianos on two Kurzweil MicroPiano modules.

I set my SMPTE offset, and started the video tape. Up to that point, I had no idea how the tempo of the music, which was essentially arbitrarily set at 133bpm, and the speed of the film, which Posner had determined was 20fps, would fit together. In fact, it fit together far better than I could have hoped — it was truly uncanny how well the visual rhythms and the musical rhythms meshed. I knew we were on to something, and when I sent a video with a rough mix on it to Posner, he was ecstatic.

Sound & Vision — Seeing Film & Music Together

A rare opportunity to view the Ballet Mécanique film accompanied by a live performance of George Antheil's score is scheduled to occur on November 13, at the annual Percussive Arts Society International Conference, which takes place in Columbus, Ohio, in the US. Performing the piece will be a percussion ensemble from the Peabody Conservatory of Music in Baltimore, Maryland, conducted by student Julian Pellicano. Yamaha will once again be providing eight Disklaviers and two digital grand pianos (for the live pianists) for the performance.

Since human players can't play the music at the tempo I used for the recorded film score — 133bpm — I had to make further cuts in the score, which would allow them to play the bulk of the piece at 115bpm, and some of the more gnarly parts at 90bpm. In performance, the sequence controlling the player pianos and the conductor's click track will be sync'ed to the videotape of the film, using a SMPTE timecode track on the video. I plan to use a Macintosh iBook for the sequences, with a Mark of the Unicorn MTP-AV USB handling the MIDI stream, and an M-Audio USB Quattro playing the audio samples and click.

Meanwhile, if you would like to view the Léger film, the Tate Gallery in London currently has an exhibit in which the film is displayed continuously from a laser video disc, although with no music as of yet. At the time of writing, a screening of the music and film together is planned to take place at the National Film Theatre on London's South Bank in August next year (2003), as part of the Unseen Cinema film festival.

Cutting Remarks

Even at the faster tempo, the music was quite a bit longer than the film, so now I had to cut it down. This was potentially the trickiest part of the process. How do you edit someone else's score, who's not around to ask, without compromising the integrity of the music? Well, it helped that I knew the piece very well, since I had entered every single one of the notes — over 400,000 of them. Antheil had written in an article that there was no formal structure to the work, that it was just blocks of things one right after the other, but I knew that was not the case: themes came back, there were soft sections and great climaxes that created tremendous tension, and an overall arc to the piece that I thought needed to be preserved. But it was also obvious that a lot of the music was repetitious and could easily be trimmed. There was also a huge section at the end that consisted of short bursts of piano trills interspersed with long silences. Although very dramatic in concert, this section would not work at all with the film, which kept its frantic pace going mercilessly from beginning to end.

By slowing the tempo 0.1 percent to 132.87bpm, a good match between audio and MIDI was achieved.By slowing the tempo 0.1 percent to 132.87bpm, a good match between audio and MIDI was achieved.

I did my initial pass at editing using a purely MIDI sequence, turning off the audio tracks, and instead playing the player-piano parts on various synths in my studio. This way, I could concentrate on how the cuts would work musically, without worrying about making the audio edits sound just right. Using the printed score and my 'hit point map' as rough guides to where in the film I thought particular musical points should land, I worked backwards from the end, checking after each edit how much more time I needed to take out. I also wrote down the bar numbers I was eliminating, since I was going to have to do this again with the audio tracks.

I didn't try too hard to make the hits, since I knew that the original filmmakers weren't thinking in those terms: the work was part of the Dadaist movement, in which events were applauded for their randomness. Nevertheless, I was very surprised how many of the hits I was able to make — if not right on the money, then within a second or so.

Surprisingly, this part of the process went very smoothly and took only a few days. In all I made 16 edits, mostly at bar lines but sometimes getting inside one of Antheil's strange time signatures and cutting off a section at a half-beat. The edits ranged in length from a single measure to 150 measures, and when I was done, a total of 343 measures, out of the original 1240, lay on my virtual cutting-room floor.

Now it was time to edit the real audio tracks. In order to ensure that the MIDI tracks and the audio stayed in sync, I locked my Pro Tools system (which was handling the audio within Studio Vision) to word clock coming from a Mark of the Unicorn MIDI Time Piece AV, and locked the sequencer to the SMPTE timecode also being generated by the MTP AV.

But something was wrong: the MIDI and audio drifted apart slightly, with the MIDI tracks ending up about two beats early at the end. I could see clearly where the last audio event took place, and so I lowered the tempo of the sequence slightly so that the last MIDI note matched the last audio event visually. The 'correct' tempo turned out to be 132.87 — one tenth of one percent slower, which is the same as the difference between 30fps (the often-presumed rate of American television) and 29.97fps (the real rate of American television). I decided not to worry about it quite yet, and went ahead and cut the audio.

I found if I edited all of the audio tracks at exactly the same point as the MIDI tracks, the edits in the player pianos were quite audible. Studio Vision doesn't do crossfades easily, and with music this percussive, crossfades would have sounded mushy anyway. After some experimentation, I found the answer. I was mainly using the tracks from the two pairs of stereo mics, one close and the other further away, and if I made the edit on the distant tracks about a half a beat later than the edit on the close mics, the 'smear' this created covered the edit beautifully, without softening the rhythm.

But now I had another problem: due to the tempo change I had made, the music had become about a beat longer than the film! I needed to change the speed of the audio and MIDI tracks, but I had to make sure that they stayed perfectly in sync, and I wasn't at all sure that Studio Vision's time-expansion DSP was going to do that. So instead, I mixed the whole piece down to stereo, at the slightly slower tempo, running my synths, samplers, and Pro Tools outputs through my Yamaha 01V mixer. Then I brought the finished track into Pro Tools, applied a little compression through Waves' L2 plug-in, and time-compressed it by 0.1 percent. It now matched the video. Or so I thought.

It was then that I noticed something funny about my MIDI Time Piece AV: it was losing all of my carefully worked-out settings every time I turned it off. I realised that the lithium battery backing up the internal memory had died. I had no trouble finding a replacement, but I began to think that some of these lost settings, which I hadn't been paying attention to, might have affected what I was doing.

The stage layout for the live performance of Ballet Mécanique, showing the mic positions and configurations. Comparing this diagram with the one below, you can see from which mics the signals on the various tracks in Studio Vision came from.The stage layout for the live performance of Ballet Mécanique, showing the mic positions and configurations. Comparing this diagram with the one below, you can see from which mics the signals on the various tracks in Studio Vision came from.Sure enough, as I went through the unit's pages, I discovered that the internal word clock generator's frame rate defaulted to 30fps, not 29.97. That's why my MIDI and audio were off — the MIDI tracks were running at 29.97, but the audio was at 30! If I reset the clock rate to 29.97, and ran the video again, my mixed audio tracks were now too long.

I went back to my first mix (fortunately I had saved it), and checked it against the video: it was spot on. So I ran it through L2, and that was the mix I kept.

A Formal Marriage

Now it was time to put the picture and audio together. The music wasn't going to be transferred to film, it was going to go to video — even though all of the venues where the Ballet Mécanique was going to be played were quite capable of projecting it on film.

There were two reasons for this, as Bruce Posner explained to me. First of all, when sound came to film in the late 1920s, the picture had to shrink slightly within the frame to make room for the optical soundtrack. As this was a silent film, the picture was larger, and there was no room on the print for a soundtrack. More importantly, however, Posner wanted to preserve the fidelity of the music, something that he thought would be impossible to do on an optical film track.

The plan, therefore, was to put my audio onto Beta SP — which Posner considers the best video format for large-screen projection — and use that videotape as the 'master' which would be copied and shown at many of the venues. I was sent the Beta SP video master, and took it, along with my Pro Tools files on a CD-ROM, over to the home studio of a friend, Richard Bock, who has done a lot of audio post-production for PBS and other major networks. He didn't own a Beta SP deck, but he was able to borrow one for a day. I was worried that the audio and video wouldn't lock, and that my MTP AV troubles would somehow infect the Pro Tools session and the DAT, but when we loaded in my files, found the right SMPTE start time on the videotape, and then went right to the end, the sync remained perfect.

A screen from Studio Vision showing the audio tracks recorded for Ballet Mécanique. The first two audio tracks derive from the close stereo mic pair in the diagram above, the second two are from the more distant pair, and the rest are from single mics on the four groups of Disklaviers. Making cuts in the audio tracks worked best when the different tracks were edited at slightly different points, allowing them to 'smear' and making the cuts less obvious.A screen from Studio Vision showing the audio tracks recorded for Ballet Mécanique. The first two audio tracks derive from the close stereo mic pair in the diagram above, the second two are from the more distant pair, and the rest are from single mics on the four groups of Disklaviers. Making cuts in the audio tracks worked best when the different tracks were edited at slightly different points, allowing them to 'smear' and making the cuts less obvious.

Back To The Drawing Board

In May, 2001, before Posner took the Ballet Mécanique on the road, we premiered it at a concert at Brandeis University, just outside Boston. The college was presenting an 18-hour 'electronic music marathon' which lasted from 9am until 3 the following morning. The Ballet Mécanique tape was shown twice, at 1pm and 7pm, and both showings garnered some of the largest crowds of the day. The Boston Phoenix's music critic called the synchronisation "brilliant". One avant-garde video artist was heard muttering as he left the hall, "I don't have to do this any more. They did it all 75 years ago."

In June, the film had its official premiere at the Moscow International Film Festival. But just before he left for Moscow, Posner surprised me by saying he wanted to show the film print, rather than a videotape. "The difference in picture quality is huge," he said. "Wait 'til you see it on film." But how, I asked him, was he going to sync the sound with the picture? His reply: "Give me an audio CD of the score to take with me, and I'll rehearse with the projectionist so that he hits the 'Play' button right at the first frame!".

I wasn't happy, and when Posner emailed me from Moscow after the premiere, I was even less thrilled. I wasn't surprised to learn that the projectionist had missed the first frame, and started the CD after the titles. The music had lasted about nine seconds longer than the film. According to Posner, nobody noticed, but I cringed at the thought. So much for sync, and all those amazing hit points...

When he got back to the US, we talked more about this new problem. I consulted a film-projection specialist who told me that a contraption could be built that would generate SMPTE timecode using the projector's shutter gate as the sync reference. The audio could then be played from a digital multitrack, timecode DAT, or even a computer locked to the SMPTE. It would be very expensive, he said, and would have to be set up carefully at every venue. I reckoned it might work at one or two museums where I could be present but, even if we could afford it, to send it around the world without someone to supervise was asking for trouble.

I suggested to Posner that if he wanted to continue showing the Ballet Mécanique on film, he should make a new print and shrink the picture so there would be room for an optical stereo soundtrack. But, he pointed out, that would be getting into a whole new set of problems: Would the sound system be adequate in each location? Would every projector play the soundtrack correctly? On the other hand, anybody can set up a system to play a CD, so playing the music that way would mean that it would, at least, always sound OK.

So our dilemma boiled down to this: either we could play the videotape, and have everything in perfect sync, but the picture wouldn't be optimum; or we could play the film accompanied by the CD, and the picture and sound would both be fine, but they wouldn't be in sync.

In the end, we decided that some venues would get the videotape, and some would get the film. But I couldn't help feeling a little disappointed. After all, it had been 77 years since Léger and Antheil first struggled with this problem, and we still hadn't got it licked!

The US premiere of the film print took place at the Whitney Museum of American Art, on September 7, 2001. The museum rented a huge sound system for the occasion, especially considering the theatre only seated about 250 people. The projector they have is capable of extremely precise speed control, and we were able to set it at 20.0 frames, and hope for the best. I started the CD by hand, knowing exactly when the first frame of the film would appear (there is no countdown, as with modern films, so I had to base this on my knowledge of the dirt patterns on the leader!) At the end, the music lagged behind by about half a beat. Not bad at all — and the effect on the audience was stunning.

Unfortunately, the event didn't get much in the way of press coverage. Four days later, the media, in New York and all over the world, found themselves concerned with much more important news. But despite the new political climate, the tour of the Ballet Mécanique has continued on schedule, and has been delighting audiences around the world. I hope you get to see it — it's really something.