Somers's support may have played a key role in keeping Mass.Illusions' team — including visual effects supervisors Joel Hyneck and Nicholas Brooks, line producer Donna Langston and software creators Pierre Jasmin and Pete Litwinowitcz — on the project long enough to prove that their untested strategy for the Painted World's effects would work. Others had posited a more traditional approach, shooting the actors against a greenscreen and comping them into digital landscapes created via multi-plane or 3-D matte paintings. Though that method might have worked, it would not have satisfied Ward's demands for artistic freedom and a sense of reality. "Vincent wanted Heaven to be a real place, as absolutely real, complex and dynamic as the world we're used to," Somers offers. "Those more traditional processes would have restricted the amount of detail and complexity we could later get into those shots, as well as the cinematography. Vincent wanted to shoot in a very normal, naturalistic style; he didn't want those shots to feel mechanical. What Mass.Illusions brought to the picture was their ability to say, 'Go on location, shoot any way you want to, and we'll use the actual morphology and the vegetation in the plates as the basis for our effects.' But as we went through some arduous times, I had to remind people that there was a reason we chose Mass.Illusions. I felt they hit the nail on the head in terms of finding ways to create what Vincent wanted in the Painted World."
Having discarded motion control and greenscreen, Somers's team relied on extensive motion tracking and Lidar surveys of the locations to re-create camera motion and locate objects so the shots of the Painted World — complex, fluid camera movements captured in Montana's Glacier National Park — could later be immersed in running pigment. "We had to do some pretty amazing tracking to match those camera moves," Somers admits. "When we wanted to alter the topography and add 3-D objects, we made Lidar surveys at nights after the production was done shooting. Lidar is basically a radar survey, typically used in building dams and so on, which gives us a cloud of points that is an actual 3-D wireframe of the landscape. Then, by doing a traditional survey of the markers we placed on location, we could line up this wireframe with the shot and basically re-create both camera movement and topography."
Amazingly, every shot in the Painted World began as a photographic plate, yet ended up looking like a fresco freshly hand-painted by a 19th-century master. However, because actors Robin Williams and Cuba Gooding Jr. had to remain 'normal,' their figures had to be painstakingly extracted from every shot. With the use of traditional tracking and the new system created and independently owned by the group (dubbed "machine vision" and based on state-of-the-art Computer Vision technology), the effects team was able to track the motion of the objects within a scene, and thus create motion maps. Scenes would then be further deconstructed into 15 to 30 mattes, then treated with particle-system brushstrokes to conceive the Painted World effect. Of course, applying dripping, swirling paint to thousands of flowers and leaves demanded both procedural and manual control. "We broke down each scene just like a painter would, deciding where we'd just throw a wash or add detail," Somers suggests. "Nature did not afford us the color balances typical of the paintings or styles that served as our references. We found that we needed to heavily, selectively grade our plates first before applying the strokes to achieve the feeling of a painting."
Beyond providing accurate camera-motion data, the "machine vision" software also yielded animation data describing the natural motion of the landscapes' vegetation and water surfaces, which enabled Mass.Illusions to apply their particle-system paints to petals and leaves blowing in the wind. "The application actually gave us motion graphs of individual elements — a particular flower or tree — so we didn't have to re-create any animation," Somers explains. "The complexity of the landscapes gave an incredible depth and naturalism to this effect. The natural animation of the vegetation lent a believability to the environment that would have been virtually impossible to create strictly on the computer.
[ continued on page 4 ]