The Virtual World: Stages of Color on The Book of Boba Fett
The streaming series The Book of Boba Fett illuminates an aspect of volume-based virtual production that is not yet widely understood: the importance of an even closer collaboration between cinematographers and color experts.
David Klein, ASC, who shot most of the Disney Plus/Lucasfilm Star Wars show’s first season (with Dean Cundey, ASC and Paul Hughen, ASC contributing their talents as well), says that when he is asked by other filmmakers to describe what working in an LED volume is like — and he is asked often — he advises them to “conceive of it as the most wonderful, complicated and challenging final-color suite you’ve ever been in. You’re coloring during the virtual scouting and previs, before you even shoot, and then you’re coloring in real-time during production.”
And even with all that preproduction and on-set work happening up front, the final color grade is as important as ever.
Preproduction Concerns
Productions destined for LED volumes must undergo a time-intensive prep in the virtual art department (VAD), where real-time animation tools such as Unreal Engine can facilitate robust, detailed lighting simulations. Narwhal Studios (previously known as Happy Mushroom) has been one provider of VAD services for The Book of Boba Fett. “We do virtual prelights within the VAD and essentially create a LUT for each scene with the cinematographer, production designer and virtual gaffer,” says Narwhal founder and CEO Felix Jorge. “We apply a post-processing volume within Unreal Engine so that every virtual scene matches what it would look like as a final piece of content.”
The VAD can not only simulate the virtual environment as it will appear on the LED screens, but also depict the photometric values of real cinema lights within the volume. “The DP guides the look with the virtual gaffer,” Jorge says. “They’ll say, ‘Let’s move this 10K or this bounce over here and bring up the values so we can see faces a bit more.’
“We ultimately generate overhead lighting diagrams for the DPs to use as a reference during production,” Jorge continues. “By the time they get to the set, the DP has worked in the virtual world for weeks, so they already have a well-developed concept of how they want to light everything.”
On-Set Workflow
On Boba Fett, Klein’s colorist team included Company 3 supervising finishing artist and ASC associate member Steven J. Scott — who was responsible for the final grade, which he performed with Autodesk Lustre as his primary software. The pair had previously worked together on episodes of The Mandalorian (AC Feb. ’20).
“The most important aspect to get right in the volume is matching the color of the image on the big screens to the color of what’s going on in the foreground,” Scott says, “and you need someone acting as a colorist on set to help make sure that’s happening. In addition, you have to ensure that the volume’s screens have a sufficient resolution to avoid moiré and pixelation. After that, it’s the same workflow as any digital capture; it’s remarkably similar to footage shot as if the foreground and background were all real on the day.”
The volume offers unique tools to assist with the cinematographer’s challenge of matching foreground and background. The ILM StageCraft LED volume used for Boba Fett, located at Manhattan Beach Studios, measures 23' tall, 75' across and 270 degrees in diameter. Though much of the screen is often devoted to 3D environments, the off-camera areas can be set to “virtual lighting cards” and other live-color-grading and lighting enhancements.
“You can do ‘virtual negative’ cards whenever there’s too much light behind the camera,” Klein says. “You can draw a 20- or 60-foot-wide window in the screen area behind you and take it down two stops, three stops — whatever you need. It’s like having live color correction available in a real-time 3D environment.
“We call them different things in the volume, but essentially they’re power windows,” Klein continues. “We had the Mandalorian character in three episodes of Boba Fett, and his silver helmet is like a giant mirror ball. If you put practical lights in the volume, you’ll see everything reflected in the armor. So instead, we created a content card that showed the environment, but we boosted it up a stop for more light. It lit him up, and we had the correct environment from the content in all reflections.”
Another member of Klein’s team was dailies colorist Jon Rocke at FotoKem. “I’m the nearset colorist, so I worked closely with Dave’s DIT, Eduardo Eguia, who worked with Dave to dial in the main look for each day,” says Rocke. “The goal was to be able to see everything clearly out of the gate, and then I built on that.
“I work with FotoKem’s NextLab system, which has a color grader built in,” Rocke continues. “We use a VFX workflow for dailies, so I start with the raw High-Density Encoding [HDE] Arri files at approximately 60 percent compression. Then, I apply an input LUT to convert it to [ACEScct], add the show LUT, and then grade on top of that with the ASC CDL.”
The ability to see the entire final frame on set via the LED volume is perhaps the greatest leap forward from the traditional VFX workflow, wherein the cinematographer typically lights foregrounded actors and elements against a greenscreen background that will be replaced in post. Says Klein, “The wonderful thing about working in the volume is that you’re grading your background in conjunction with your practical lights on set.”
Postproduction
Scott notes that Boba Fett involved essentially the same shaping, contrast and color adjustments as material created on actual sets or locations — such as the marrying of background and foreground elements, and making secondary adjustments to the background and foreground independently.
“While effects shots that use greenscreen often need quite a bit of massaging in the grade to bring all the elements into the same space,” Scott says, “I found that shots that were done with virtual sets often only need the type of adjustments that carefully lit live-action material does. We’re still going to go through and isolate portions of the frame to adjust color, density, contrast, and all the variables we always fine-tune in the grade, but if the DP and the whole team are involved in setting up the volume and matching the lighting is as talented and prepared as they were on this show, we could essentially treat the shots the way we would actual location photography.”
He adds that, just as in traditional productions, the necessities of the editing process sometimes led to more involved work in the finishing suite. On some occasions, he says, “there was every intention to arrange shots one way initially, but things changed editorially to accommodate story additions like flashbacks, etc.” He offers an example of “a sunny shot that’s cutting next to something with more clouds — you need to reconcile those differences just as you would in any classic DI situation. One of your options might be a simple color correction to brighten the darker shots to match. But other options include drawing some shapes that look like shadows on the ground being cast by a statue or vehicle.”
Describing a type of LED-wall-specific circumstance that called for a bit of additional attention, Scott says, “You can’t have brighter highlights on the helmet than you have in the sky, because then it’s going to look like a composite and not an in-camera effect. The background itself ultimately tells you what you can do.”
Collaboration in Virtual Space
“The more the cinematographer and colorist know about each other, the better a final product we can deliver together,” says Klein — who, with approximately 150 days of shooting in LED volumes, is currently one of the most experienced cinematographers in that milieu. “We’re constantly converging production and postproduction. For a cinematographer working in the volume, the final image is closer to what’s in your head than ever before.”
