Boldly Going Virtual with Star Trek: Strange New Worlds

All photos courtesy of Paramount Plus

Star Trek: Strange New Worlds is technically not the first Trek project to embrace LED-volume in-camera visual effects — that achievement belongs to Season 4 of Star Trek: Discovery — but the new Paramount Plus series is the first in the franchise to be designed entirely around them. 

Strange New Worlds, the latest series in the Gene Roddenberry spawned sci-fi universe, serves as a prequel, chronicling the adventures of the crew aboard the USS Enterprise before James T. Kirk took command. A spinoff of Discovery (AC July ’21), Strange New Worlds features the famous starship commanded by Captain Christopher Pike (Anson Mount) and crewed by familiar characters such as Spock (Ethan Peck); Nyota Uhura (Celia Rose Gooding); and Una Chin-Riley (Rebecca Romijn), aka Number One. The first season was shot by cinematographers Magdalena Górka, ASC, PSC and Glen Keenan, CSC; the latter’s contributions included shooting the show’s pilot episode.

The Star Trek franchise launched in 1966 — with what is now known as Star Trek: The Original Series (TOS) — and has always been at the forefront of visual effects. After TOS went off the air in 1969, the story continued through several theatrical features and TV spinoffs. Each successive show took advantage of its era’s cutting-edge visual-effects techniques, such as computer-aided motion control and digital compositing (as used on Star Trek: The Next Generation) and leaning hard into CG animation (beginning mid-series on Deep Space Nine, and from the very start with Voyager.)

So, it’s only logical that Star Trek and virtual production come together now.

Górka, who shot five episodes of Strange New Worlds, was delighted to take on the project. “I’ve always had a major interest in science-fiction, and to be able to create new visuals for such an iconic franchise was a huge honor,” she says. “Solaris, Arrival and Ex Machina were all huge inspirations.

“Before we started, we just didn’t know how the volume would work in terms of lighting,” she continues. “My friend Jose Parra from FotoKem put me in touch with Greig Fraser [ASC, ACS], who [was instrumental in developing] the volume for The Mandalorian. Greig was very generous with his time and relayed to me that he wants this technique to be successful, so he was happy to share the knowledge. I learned that the most amazing thing about it is that you are lighting with the wall, so it’s important to keep that in mind while building assets and to design lighting directly into the screen content as a starting point. Building assets was my favorite part of the job. Lighting amazing designs created by production designer Jonathan Lee and Pixomondo — seeing how it comes to life — was extraordinary. We were encouraged to go bold, and we had significant creative freedom to shoot a very colorful and slick show. Our imagination was the limit.”

Górka used Arri Alexa LF cameras with a custom set of Cooke Anamorphic /i 1.8x Full Frame Plus Special Flair lenses.

Production for Strange New Worlds takes place in Toronto, Canada, with the show’s substantial LED-wall work performed on visual-effects house Pixomondo’s stage (which, as with all of the company’s Canada-based LED-wall stages, was built in partnership with Canadian production rental company William F. White International Inc.). The stage is a 70'x30' horseshoe-shaped volume that uses Roe Visual’s Black Pearl BP2 2.8mm LED panels for the walls and Roe Carbon Series CB5 5.77mm panels for the ceiling. The panels are fed by Brompton Tessera SX40 12-bit, 4K LED video processors, which output real-time animation from Unreal Engine. The volume — which is also used for Discovery’s LED-wall work — includes OptiTrack motion-capture cameras to handle multi-camera tracking.

The Pixomondo virtal production team includes virtual-production and visual-effects supervisor Nathaniel Larouche and executive producer Paolo Tamburrino. “Virtual production with LED volumes is a new toolset for filmmakers to create their stories differently, from day one,” says Tamburrino. “The actors are not just surrounded by the wall, but they can engage and interact directly with its contents.”

Adds Larouche, “There’s something tangibly better about capturing these shots in front of a real lens versus compositing everything in post. The natural depth of field and the way the light falls on fine details like hair and cheekbones — all of that definition and accurate lighting are what you’re losing when removing greenscreen spill to get a good composite. Using the LED volume gives you back all of those highlights and creates a beautiful and natural-looking image that we’re all very proud of.”

Referencing an interesting use of the LED wall that allows an actor to essentially serve as their own double within a scene, Tamburrino recalls “a sequence where a character has to jump out of the cargo hold. We shot the actor’s jump on a bluescreen, and then keyed and placed the imagery on a card — a 2.5D technique — within the Unreal environment. Later, in the LED volume, the actor performed the scene up until the moment of the jump. Then, a custom control allowed us to trigger the playback of the element in line with the actor’s performance/cue,” so the actor’s live performance was able to lead directly into the VFX shot of the actor, now appearing on the LED wall, leaping from the cargo hold. Tamburrino adds that the team is “looking for ways to integrate more digital doubles into future environments.”

In addition to various extraterrestrial planetary environments for the Enterprise crew to explore, the LED volume was incorporated into several standing ship sets. These included the engineering room, mess hall and cargo bay — familiar locales that have mainly been depicted in previous Trek shows via physical set builds or bluescreen shoots.

Because so much of the success of LED in-camera VFX depends on camera-ready environments, the Strange New Worlds team spent much longer in preproduction refining virtual sets — with close collaboration between Pixomondo’s virtual-production team and the art department, which led to the creation of the company’s dedicated virtual art department (VAD) for Season 2. For Górka (who worked exclusively on Season 1), this meant new ways to interact with production design. “We could view and walk through [virtual versions of sets, before they were physically constructed], in VR using Unreal Engine,” she says. “We were able to see every corner of the set in 3D and think about how to light it before our lighting sessions. Then during our lighting sessions on iPads with Pixo, we would add specific sources and colors, which would be part of the design, into the sets. We would discuss in detail what kind of lighting I wanted to see in the virtual environment, and where it needed to be placed. We usually avoided direct sunlight, as it was still challenging to emulate the effect in the volume.

“We’re constantly trying to infuse more dynamic lighting and moving, interactive elements into our environments. It’s terrific for a DP to have access to tools like these,” Górka continues. “It was a long journey from my film school in Poland, where I learned how to light with just two 650-watt lights and candlelight, to the place where I can use cutting-edge technology to perfect my craft. I was very grateful for the experience, and for being able to create on this level. It gave me great knowledge for the future.”

Tamburrino notes that the lessons learned during Discovery’s first season with an LED volume carried forward to Strange New Worlds. “Our approach has evolved throughout production,” he says. “We started adding new features, like subtle animation in the real-time environments. For example, for the engineering room, we’ve developed ‘energy’ that’s flowing inside the conduits and warp core. A more complex effect, in combination with custom-built board controls, was achieved in the alien chamber, where interactive hieroglyphs could be controlled within the virtual environment — both intensity and colors. We’ve also added DMX integration, so the lighting operator can start controlling certain key lights that were custom-built into the associated virtual environment and routed to the individual DMX channels. We’ve had significant progress over the past 12 months.”

Avid Trek viewers know that key plot developments often hinge on the action occurring in the engineering section of the Enterprise. While this futuristic and expansive set had been shot at the Anheuser-Busch brewery and the National Ignition Facility for the respective J.J. Abrams-directed features Star Trek (AC June ’09) and Star Trek Into Darkness (AC June ’13), for Strange New Worlds the iconic setting was created by combining practical foreground consoles and guard railings with an Unreal Engine-powered LED-volume environment that showed off the mighty inner workings of the ship’s warp core.

“We had 250 or so little virtual light fixtures in the engineering-room design, and 60 real light fixtures — which included LED strips, LED monitors, Arri SkyPanels and Astera Titan Tubes — in the physical set, in the foreground,” says Górka. “We would always do a ‘blend day’ in the volume when a set was close to production. It’s essentially a live color-grading session, where you balance the real physical elements in the foreground with the virtual environment. The wall was often too bright — so first we’d bring it down, and then balance and color-correct all the foreground and volume fixtures. We could also add animation, like electricity arcing, flashes, smoke and other elements, into the wall. Most of them had to be pre-designed, but some of them, like moving smoke or nebulas, could be done on the fly.”

Larouche adds that engineering was “one of our most challenging sets, but it made us realize that having extremely dynamic and moving elements makes things feel much more alive and realistic. We’re constantly trying to infuse more dynamic lighting and moving, interactive elements into our environments. The DPs love it, too, because they want to be able to mix up the look on the fly as we’re shooting, instead of being tied into a single style.”

“One thing we’re always thinking about is constructing even larger LED volumes,” says Tamburrino — who began work on Season 2 of Strange New Worlds before the first season had even aired. “That would give our directors more chances to frame upwards toward the ceiling — because, right now, it’s likely to require a digital cleanup if we shoot off the edge of the screens, due to technical limitations, such as color shifts and visible edges between the wall and ceiling. Also, [since we have] the flexibility to add SFX — [such as] smoke and sparks — and practical/traditional lights into the ceiling, a benefit [to a larger volume] would be to keep SFX and lighting rigs out of the camera line. We also want to create more interactive animation, such as having a character carry a real-world flashlight, which affects the virtual environment.”

Says Górka, “It was a huge privilege to contribute to the Star Trek legacy.” Regarding the new technologies, she adds, “These tools are amazing because it allows you to have no boundaries to your imagination.”

You’ll find much more of AC’s past Trek coverage here — going all the way back to the original series.