Discover Magazine August 1984
Lights, Cameras... Computers
By Stuart Gannes
With its sleek black face, glowing video screens, and Darth Vaderesque controls, the device could easily be mistaken for a prop in yet another sequel to Star Wars. In fact, it springs from the fertile imagination of George Lucas, creator of the Star Wars trilogy, who calls it EditDroid. As connoisseurs of Lucas's movies know, 'droids - short for android, a word with Greek roots meaning automaton with a humanlike form - are the playful robots that populate his highly successful space operas. Yet while it is gifted, EditDroid is one robot that will never star in a Lucas film.
Instead, it will help make one. Behind EditDroid's curving console lies a computer almost as smart as any in Lucas's flicks. With its powerful memory and infallible logic, Lucas thinks, he will be able to revolutionize one key aspect. of moviemaking: film editing, the job of assembling the raw, uncut celluloid into smoothly flowing, polished scenes. Until now, film editors have had to sit before a viewing device - usually a miniature projector called a Moviola. As the film reels past, the editor cuts out sequences, adds others, or simply shuffles snippets of film around, as if rearranging a manuscript.
It can be a daunting task. A typical two-hour feature consists of six reels, often more than 10,000 feet of film, which has been culled from footage many times that, length. By the time the film is edited into its final form - a job that often takes as long as the shooting, itself - the editor may have made as many as 2,000 splices, in which snippets of film are glued end to end.
EditDroid vastly simplifies the task. Seated at its controls like Luke Skywalker piloting his spaceship, the film editor can rearrange footage almost as easily as a text editor at a word processor can juggle words and phrases - inserting and deleting frames, taking a sequence from one place and shifting it to another. An additional benefit: he can instantly see on EditDroid's central video monitor what his handiwork will look like in a theater.
TOP
Lucas's automated editor was unveiled at the National Association of Broadcasters convention in Las Vegas in May, where it was the talk of the show. But EditDroid is only the first in a trilogy of products that Lucas hopes will help bring the art of film making, into the age of electronics.
One of the other two devices is an audio signal processor, or ASP, a far-out version of the sound mixers familiar to stereo buffs. ASP does for sound tracks - the band of squiggly lines at the edge of a film that encode the film's sounds - what EditDroid does for images: it uses its silicon circuitry to mix, edit, and even synthesize the music, speech, and sound effects so important to film makers, especially somebody like Lucas for whom sound is as important as sight in stirring the emotions of an audience. ASP has already been used to heighten the drama - by changing the pitch of a scream, for example, to make it more chilling - in the current hit Indiana Jones and the Temple of Doom, co-produced by Lucas and Steven Spielberg.
The third Lucas-inspired creation is a computer graphics machine, called Pixar, which uses the digital magic of the computer to produce images and scenes, even backdrops. Traditionally, Hollywood has relied on animators or modelmakers for these. But a few minutes of animation can require thousands of individual drawings. By contrast, Pixar's graphics software can in almost a flash create amazingly realistic images of everything from trees and mountains to the eerie landscape of a distant, lifeless planet.
Of the threesome, EditDroid (cost: $87,000) is Lucas's favorite. The 40-year-old director, writer, and producer began his career as a film editor, and still does much of his own editing. He knows well the frustration of splicing miles of film by hand - the exasperation of encountering torn sprocket holes in some crucial footage, or of having a splice fail when the director or producer is clamoring to see the finished product. Says Lucas, "Anybody who's worked with film realizes what a stupid nineteenth century idea it is."
By 1980, with millions rolling in from the second Star Wars epic, The Empire Strikes Back, Lucas decided to do something about bringing film making into the twentieth century - and beyond. As he explains, "Studios are not willing to put up the ten or fifteen million necessary to develop computer technology for film making. Somebody has to do it. The whole industry should be doing these things. Why they don't is beyond my comprehension."
TOP
At the campus-like headquarters of Lucasfilm Limited in San Rafael, California, north of San Francisco, he began assembling a staff of computer scientists. Now totaling more than 70 people, many of them with Ph.D.'s, this cadre of programming and hardware experts operates in an atmosphere of secrecy that would seem appropriate for the Pentagon. But DISCOVER has been given a rare glimpse inside Lucas's multimillion-dollar computer think tank, where scientists and engineers clad in blue jeans work in conditions of corporate elegance: plush hunter-green carpeted hallways and woodpaneled offices, decorated with oak furniture and leaded glass windows.
Few of Lucas's disheveled collection of computerniks knew or even cared much about film making before he recruited them. Ed Catmull, the 89-year-old head of the computer development group, came from the New York Institute of Technology, a leading center for computer-graphics research. So did Ralph Guggenheim, 33, director of the EditDroid project. "The goal was vague at first," he says. "Just apply computers to film editing and the creation of special effects."
After a year-long study of Hollywood production techniques and equipment, Catmull proposed the computerized editing machine, as well as the two other devices, both still in the experimental stage. Before EditDroid can perform its electronic magic, the unedited footage must first be scanned by a video camera and etched by lasers onto video discs. Each disc can hold up to 80 minutes of film. Instead of sorting mechanically through reels of film, the editor simply taps out instructions on EditDroid's console to call up whatever sequence he wants.
In three to five seconds, the machine can locate any frame or sequence of frames on the disc and begin flashing them onto the previewing monitor. If the editor wants to use those particular frames, he marks the beginning and end - the head and the tail of the clip, in movie jargon - by appropriate control-pad commands, and the computer stores their position in its memory. When the editor finds the place where he wants to insert them, he simply calls them back and commands EditDroid to add the footage to the sequence of frames already compiled.
TOP
When the editing is completed, EditDroid's list of marked frames is turned over to a film laboratory where the actual film, rather than the low-resolution video disc version of it, is copied in the edited order. With EditDroid, says Guggenheim, "the editor can think about the story he is trying to tell and not worry about the chores of editing."
Lucasfilm's sound mixer is even more ambitious technologically. Today's films have as many as six sound tracks to drive the multiple speakers in many theaters. But even these are usually the distillation of many more tracks. Project chief Andy Moorer, 39, points out that in a film like Return of the Jedi the sounds in a typical scene may represent a mix of 70 separate tracks of dialogue, music, and audio effects. A single change in one of the original tracks - say, the boom of a rocket or the pitch of a siren - would require remixing all of them. Jedi needed only five film editors but 17 sound editors.
"What we set out to do," says Moorer, who has a Ph.D. in computer science from Stanford, "Was to put a computer in the middle of all of this." That was not easy: just as the visual images must first be converted to electronic signals for EditDroid, so the sounds must be turned into digital form for ASP. This means every flutter of noise has to be translated into the "on-off" binary language of the computer. In other words, the sounds become numbers.
The system is not very economical. A single spoken word may have to be represented by a thousand or more digits. Handling such a flood of numbers requires enormous computational power. Yet ASP can perform as many as twenty million arithmetical calculations per second simultaneously for each one of the film's six audio tracks. This prodigious mathematical ability rivals that of many supercomputers.
In addition to being a numbers prodigy, ASP is quite garrulous. It can synthesize speech, the sounds of musical instruments, and even special effects by the same mathematical techniques. In Indiana Jones, for example, there is a hang-onto-your-seat scene in which Jones and his pals, while dangling precariously from a rope bridge slung across a deep chasm, come under attack by a band of archers. Lucasfilm technicians had recorded the sound of a flying arrow in a studio, but they discovered that the whistling noise did not last long enough to match the flight of the arrow on the film.
ASP came to the rescue. Moorer copied 25 milliseconds from the middle of the one-and-a-half-second recording and spliced the duplicate sounds to both ends, all electronically. Then he manipulated the arrow's noise so that it faded as the missile moved from left to right across the screen. To ensure total accuracy, Moorer even used ASP to include a Doppler shift - the change in pitch from high to low heard when an object sweeps rapidly past. Thus, as the arrow flies by actor Harrison Ford's head the audience hears a subtle change of frequency in its noise. In this way the sound track dramatically increases the audience's sense of the hero's peril.
TOP
The third Lucasfilm machine was designed to create new worlds. Pixar, built under the direction of Catmull and Alvy Ray Smith, 40, another recruit from New York Tech, produces extraordinarily complex and lifelike graphic simulations that rival and sometimes exceed those born of traditional animation. Pixar's secret lies in its software: intricate algorithms, or mathematical models, that catch the essence of' real forms and textures in a series of digital instructions that control every element, or pixel, of the picture. A conifer tree, say, requires an algorithm that draws branches progressively smaller as the trunk rises from the ground. Still more algorithms are required for realism. "If the branches were placed evenly around the trunk," Catmull explains, "they would look funny. A real tree has a certain randomness to it, so you write a program called a branch placer that has a randomness factor built into it."
The Lucasfilm team has developed algorithms that draw mountains, clouds, even the blur of motion that might follow a wave hitting a beach or two billiard balls colliding. The mathematical underpinnings of this work are almost as far out as some of Lucas's films. Mountain slopes, for instance, are drawn with the help of fractal geometry - a branch of mathematics involving a repetition of forms or irregular shapes, recently codified by IBM researcher Benoit Mandelbrot (Discover, June 1982). To match the quality of 85-millimeter film, Pixar uses as many as 16 million pixels for a single frame. Once the computer image is complete, Pixar's laser scanner reads the image pixel by pixel and exposes it onto 85-millimeter film.
For all their emphasis on realism, however, Pixar's designers are hardly slaves to it. In true Lucas ' spirit, their objectives are sometimes otherworldly, as in the case of a 60-second animated sequence commissioned for Star Trek II that shows life beginning on a formerly lifeless planet. This "Genesis demo" is widely recognized as one of the most sophisticated pieces of animated computer art that has yet been produced. But for the computer wizards at Lucasfilm it is only a beginning. Eventually, says Catmull, "the computer will allow Hollywood to tell stories that could not have been told any other way."
At the controls of EditDroid, Lucas's new film-editing computer, designer splices movie clips electronically.
Below, a computer-generated sequence from Star Trek II shows the birth of life on a barren, moonlike planet after it has been struck by a torpedo and engulfed in flames.
IMDB Star Wars, Toy Story, Ralph Guggenheim, Ed Catmull and Alvy Ray Smith
Filmography links and data courtesy of The Internet Movie Database.
TOP
Final Cut Pro links 
