John Knoll on Rogue One’s Visual Effects: Part 1 of 2

Unit photography by Jonathan Olley, Giles Keyte and John Wilson. All images courtesy of Lucasfilm Ltd.

In May 1978 — almost a year to the day after the original Star Wars, which would later come to be known as Episode IV – A New Hope, premiered — John Knoll paid a visit to Industrial Light & Magic, the visual-effects company famously established by George Lucas. Knoll was 15 years old at the time, with a penchant for model-making and a keen interest in the nuts and bolts of the motion-picture craft. Less than a decade later, after earning his bachelor’s degree from the University of Southern California, he landed a job as a technical assistant at ILM in February 1986. “The first thing I did,” Knoll recalls, “was on the Dykstraflex stage with Selwyn Eddy and Ray Gilberti. We were shooting the hangar at the beginning of Star Tours [the Star Wars-themed attraction for the Disney theme parks]. After a few weeks I got moved into effects camera and started working on things like the hyperspace tunnel and the streaky stars, flak bursts, and things like that.”

From those beginnings, Knoll climbed the ranks to visual-effects supervisor. In that role — in addition to winning an Academy Award for his contributions to Pirates of the Caribbean: Dead Man’s Chest and earning four additional Oscar nominations — he has contributed to the Star Wars Trilogy Special Edition (AC Feb. ’97), Star Wars: Episode I – The Phantom Menace (AC Sept. ’99), Episode II – Attack of the Clones (AC Sept. ’02) and Episode III – Revenge of the Sith. Now serving as ILM’s chief creative officer and senior visual-effects supervisor, Knoll pitched the idea that would become Rogue One: A Star Wars Story to Lucasfilm President Kathleen Kennedy. The feature would go on to be directed by Gareth Edwards and shot by Greig Fraser, ASC, ACS; attached as he was to the project, Knoll acknowledges, “I just kind of declared that I would be visual-effects supervisor.” He shared those duties with Mohen Leo.

Six weeks before Rogue One’s Dec. 16 release date, Knoll spoke with AC over the phone from ILM’s headquarters in San Francisco’s Presidio.

American Cinematographer: Given that Rogue One was born from your pitch, does the project feel more personal than other features on which you’ve worked?

John Knoll: You can’t help but feel a little proprietary about the ideas and how they’re represented in the finished film. But film is a highly collaborative medium, so while it started off as a pitch that I gave to Kathy, it is the product of a great, great many people’s input now.

Gareth Edwards comes from a background in visual effects. Did that help streamline your collaboration with him?

Knoll: It provided the right vocabulary to talk with some great specificity about things that he wanted to tweak. It’s always helpful when it’s not vague — you know, ‘Something doesn’t feel right about it. See what you can do.’ We could talk about different ways of executing things, and the pros and cons of different approaches.

Rogue One’s story leads directly into the beginning of A New Hope, but it’s also the first ‘Star Wars Story,’ separate from the episodic movies of the Skywalker saga. Given that, how much freedom did you feel stylistically, and how much of an obligation was there to match the look of Episode IV?

Knoll: On one hand we’re depicting locations and showing vehicles that are directly referencing the other film, so you want them to feel like the same thing. We’re not trying to redesign spaceships that we’ve seen before; when you see an X-wing, it’s us doing our best to make it look like the X-wings that you’re familiar with. But our general mantra was, ‘Match your memory of it more than the reality,’ because sometimes you go look at the actual prop in the archive building or you look back at the actual shot from the movie, and you go, ‘Oh, I remember it being a little better than that.’

In opposition to that, in terms of the overall style of the movie, is this mandate that Star Wars Stories are not part of the through-line of [Episodes] VII, VIII and IX, so we don’t have to follow the stylebook quite as slavishly as when we’re doing additional chapters of the same story. So in terms of cinematography style, mood and tone, we’re making it a different movie than what the numbered Episodes are.

How would you describe Rogue One’s style?

Knoll: Gareth has a very documentary, vérité style to his shooting — a lot of handheld camera — and he’s pretty non-specific about the blocking with the actors. We show up on set, they know their lines, and they know from the script what’s happening, but Gareth is not going to tell them, ‘You walk in through this door, then you stand here. Then you start back there, and as you say this line, you take two steps forward.’ He’s not going to give them that level of micromanagement, because he feels like that comes across as being artificial. So he’ll let the actors do what feels right to them and then make adjustments.

It’s the same with the shot design: He’s looking through the viewfinder and finding what feels right. The first couple of rehearsals look a bit chaotic as we’re trying different things. And I think that’s one of Gareth’s great strengths, being able to react to the image in front of him when he’s looking through the viewfinder and seeing, ‘What’s the really nice-looking image here?’ That adds a wonderful immediacy and realism to this; it adds this authenticity that more carefully planned work doesn’t have. And I think that’s really fun in the Star Wars universe, that things don’t seem quite so storyboarded and deliberate. But it is a challenge for visual effects. Sometimes the first time I see the angle we’re shooting is when we’re actually shooting it, so it can be a little hard to prepare for — ‘Oh, you know, we should have some tracking marks up there,’ or, ‘It would be good if we had a bluescreen there, because the actor’s hair is blowing over something I’m going to have to replace.’

With that approach to the shoot, did you still use previs?

Knoll: We did a little bit for figuring out some of the more complicated action scenes that we had to shoot in multiple locations, and to decide what we would try to do on location and how much set we would need for this and that. But there were big parts of the film that weren’t very [previsualized] at all; we’d just go in and Gareth would find the angles and react to what looked good.

Were you on set all throughout principal photography?

Knoll: Yeah. Probably half the time we had a splinter unit going, and sometimes splinter was shooting something that was a little bit more visual-effects intensive and required somebody to help make sure that we were doing the right thing, while first unit was shooting something with a bluescreen out the window or some wires to paint out, but I didn’t really need to be looking over their shoulders during that. So Mohen Leo and I split duties; he was working out of ILM’s London office, and when there was a splinter unit going, he would come in and we would decide who would be with splinter, depending on where I thought I could do the most good.

Greig Fraser tried to stay involved in the lighting of everything, even when it was second unit or splinter unit. And often the splinter unit was shooting on sets that first unit had already shot on, so the lighting had already been designed by Greig. The kinds of things that often got thrown to splinter were things like ‘TIE fighter cockpit,’ where there’s not a principal actor. So Greig would prelight those things and then have [second-unit director of photography Barry] ‘Baz’ Idoine shoot them and make adjustments.

At what point during preproduction were you able to have your first conversation with Greig Fraser?

Knoll: Very early on. Something I try to do on all shows is to talk through some of the things that are often challenging on visual-effects productions so they don’t get planned wrong. For example, when you have a set fragment and a lot of bluescreen or greenscreen, two things tend to happen: Often camera operators don’t know how to frame the shot properly because they’re only seeing a piece of the image, and often cinematographers only light what’s in front of them because that’s what they’ve been trained to do — they’re not necessarily lighting for what the final image is going to be. A classic problem is an interior where there’s a window to daylight exterior, and instead of actually having that daylight exterior out there, there’s just a bluescreen. Those are always extremely difficult exposure splits because you want the exterior to look like it’s brighter but you still want to read actors’ faces in the foreground. Trying to find that balance is often really difficult if you can’t see the actual exterior out there. I’ve been trapped on a couple of shows with way-overexposed sunlight hitting windowsills; if I put the background out there at that brightness, it’s going to be completely blown out, and that’s not what anybody wants, so now the ratios are completely wrong.

Star Wars has always been known for pushing effects technology forward. What have been some of Rogue One’s advances?

Knoll: There are incremental things on every show you work on, but there are a couple of larger areas on Rogue One. One of the challenges on this show is we’re doing some digital-human work; we have a couple of scenes where there are characters that are human beings, but they’re digitally generated. That’s been touched on in the past, but this is a fairly demanding application of that. So we did a lot of work on facial motion-capture tools, skin shading and those kinds of things. I think that’s going to get some attention when the film comes out.

[Editor’s note: This interview took place in advance of Rogue One’s release, before Knoll could go into detail about the characters in question: Grand Moff Wilhuff Tarkin and Princess Leia Organa. For the former, the visual-effects team had to create a digital Peter Cushing, who portrayed the character in Episode IV and passed away in 1994; for the latter, the crew needed to create a new shot of a 19-year-old Carrie Fisher, as she appeared in A New Hope.]

What’s been the workflow in those instances? Was there someone on set during principal photography to represent the digital human?

Knoll: Exactly. You have to cast an actor to play that role. [Guy Henry was cast in the Tarkin role, and Ingvild Deila provided Princess Leia’s on-set performance.] That performance drives the digital character — in the same way that we did Davy Jones [played by Bill Nighy for Pirates of the Caribbean: Dead Man’s Chest and At World’s End]. It’s good to have somebody on set just like you’re shooting any other live action so the actors can play their performances against each other, the camera operators have someone to frame up on — for all those reasons.

Beyond the principal photography, was there a separate shoot to get any of the facial performance capture?

Knoll: No, the facial performance capture happened on-set while we were shooting. We did two approaches. For one character, we used an experimental new technique that was a no-head-rig version; we had standoff cameras that were on longer lenses framed in tight that were trying to get the highest-resolution imagery but without having a helmet-mounted camera.

The other character was mostly done with helmet-camera work. Other facial-motion-capture helmet systems use white LEDs to illuminate the face, and I can’t stand that because it spoils my ability to use the plate photography as good lighting reference. So one of the things we did differently was we used infrared cameras with infrared lights on them, and it meant that we could light our tracking dots beautifully so that the cameras could see them well, but it didn’t interfere at all with Greig’s lighting or my ability to use the taking camera’s plate as really terrific lighting reference.

Did you feel you got a better result from either the helmet-camera rig or the standoff long-lens cameras?

Knoll: They each have tradeoffs. The reason everyone uses the helmet cameras is that the quality of data is very high because you frame those cameras very closely, and if you have a light attached to the camera, then the dots are always visible. So you get this high-quality data, but it is worse for everybody else. The actor’s got this thing hanging off his head, and no matter how much he tries to ignore it, it’s always there in the periphery of his vision — especially with visible-light cameras. That’s half the reason to have gone with the infrared, was to try and eliminate that distraction from the actors. But then the camera-boom system itself gets in the way if two characters have to get really close to each other, and sometimes there are wardrobe reasons that make it a problem.

In Part 2 of this interview, Knoll discusses Rogue One’s CG ship models as well as the methodology employed to realize the droid K-2SO, played by Alan Tudyk.

See the February 2017 issue of AC for the complete production story on Greig Fraser, ASC, ACS's cinematography.