Convergence is good and it is literally everywhere.
It’s in the media we consume and the devices that we use to access it. The blending of different technologies is an enabler; we watch films on our phones and surf the internet on our televisions, we listen to the radio on our laptops and we eat pasta with a spork.
But there are opportunities to add value not only not in the way we consume but in the way we produce, and convergence is helping creators exploit that too.
NEP acquired Halon Entertainment alongside Prysm Stages and Lux Machina Consulting in 2021, taking a step closer to convergence across all its delivery channels. NEP Virtual Studios provides a range of enhanced services, from virtual stages and virtual production to augmented reality and visualisation. These tools not only streamline production processes and minimise post production time but enable creatives to build impossible worlds at a fraction of the cost.
Tools like these have long been part of the film industry, but virtual production techniques are also augmenting live broadcasts to help grow audiences and drive engagement.
Video Games
Chris Ferriter is the president of Halon and his career has been all about convergence. Starting in the film industry in the early 1990s he got a front-row seat for its move from optical to digital visual effects. He went on to spend 18 years in the video game industry and joined Halon in April 2016.
“Halon was already using video game technology at that point, and in 2016 it pledged to use Unreal Engine as the core component of its pipeline,” he says. “The idea was to use this technology to help creatives tell better stories. It was the convergence of two things that I loved, and seven years later the company has grown exponentially.”
The service is Unreal
Designed as a video game development tool by Epic Games, Unreal Engine now enjoys a healthy second life in film and TV, from the creation of new worlds to real-time rendering for LED volumes.
LED Volumes, a series of connected LED panels, can wrap around a film set to form a virtual background. Visuals can be created in pre-production and the physical set design matched to them. The Unreal gaming engine ties camera movement to the background to create a realistic moving environment in the same way as it would in a video game.
Running in such a pure digital environment provides a way for the talent to interact with the environment in real-time; unlike working with a green screen, it is as if they are actually there.
Virtual Artworks
“It speeds up the creative,” adds Ferriter. “We first used it on War for the Planet of the Apes and it not only sped up the creative process but opened up the doors for new services. Working this way allows the previs process to provide more information to departments right across the production.
“It also opened up the door to our virtual art department, which we went on to use on The Mandalorian to build digital sets using photogrammetry data, all run in real time on LED Volumes.”
On The Mandalorian, the LED Volume measured 21 feet high and 75 feet in diameter, and it wrapped around the film set. The designer and integrator of volume was Lux Machina, also now part of NEP Virtual Studios.
The combination of Halon, Lux Machina and Prysm gives NEP Virtual Studios access to the entire pipeline for virtual production, although this is a term which Ferriter tries to shy people away from.
“From a technical perspective the approach is still all about communication and coordination,” he says. “If you know upfront that you are going to be shooting a sequence on a virtual stage and you are going to be using in-camera visual effects, you would plan that at the previs phase, which would inform the art department of what they need to build. The way the camera moves also determines how you will build that set.
“Previs is part of the virtual production process, informing the creation of digital sets and how it will run on the virtual LED stages. It is still all about the production and we are simply using the most appropriate toolkits to better execute that production.
“Our industry has always worked this way; when new tools come along, we adopt those tools, like when we switched from optical to digital effects; in a sense, virtual production is really an evolution of rear projection and green screen methodology that has been around for a long time. What companies like Halon are doing is embracing these modern tool sets to do these things better. We integrate new tools into the pipeline, figure out how to use them in an efficient manner and generate opportunities that we didn’t have before.”
High kicking backdrops
This spirit of convergence allows companies to exploit these technologies to add value across non-traditional verticals.
Karate Combat is a case in point. In April, season three of Karate Combat’s live-streamed championship worked with NEP Virtual Studios to create a hybrid live environment which blended live-action with an immersive CGI environment. Streamed to more than one hundred territories across the world, it smashed streaming and engagement records to generate 9.4 million live Facebook views compared to the previous tournament’s 4.2 million.
Halon built the environments and Lux Machina incorporated live camera tracking, real-time XR set extension and in-camera compositing, as well as blending both environments together. Opportunities where virtual reality environments are able to augment live sports are an exciting development.
“As a broadcast project, Karate Combat was very cool,” says Ferriter. “We were tracking multiple free-space optics and a virtual environment alongside a live audience and it brought together different groups within NEP.
“This is an area where we would never have been able to work before, but these technologies enable us to add value to new verticals like broadcast. Getting a better understanding of all of the places that can benefit from these workflows is an exciting part of this journey.”
Museum Pieces
And it doesn’t stop there. Although Halon is marking its 20th anniversary with box office number ones courtesy of Cocaine Bear, John Wick Chapter 4 and Transformers: Rise of the Beasts, it is applying the same technology outside of its traditional markets.
“As well as Karate Combat, we are working with museums to capture and build digital proxies of artworks that can serve as a mechanism for preservation or to teach people who don’t have access to those artworks.
“That’s a place that you would never think has crossover with Cocaine Bear or even Karate Combat, right? But it does. The technology can stretch.”
Read Kevin’s interview with the team at Halon about their work on Cocaine Bear in the June/July issue of TVBEurope.
