UK TV facility dock10 is working with the University of York to explore how artificial intelligence can tackle simulating realistic light interactions for productions shot in virtual studios.
The project brings together AI researchers from the university and virtual production experts from dock10. It has been awarded an embedded R&D grant from XR Network+, which provides funding and support to researchers working in virtual production technologies, facilitating collaboration between academia and industry.
Live or ‘as live’ TV productions – including sports programmes, game shows and music performances – that film in multi-camera green screen studios currently struggle to achieve realistic lighting interactions.
An actor’s shadow, for example, has to be added in post production.
In single-camera film or drama productions, these effects can be achieved with an LED volume, but the technique is not suitable for multi-camera productions which need to capture multiple angles simultaneously.
The project will explore how AI can help simulate accurate light interactions between physical actors and virtual environments in multi-camera virtual studios, and aims to lay the foundations for the creation of innovative AI-powered compositing tools that ultimately will be applicable in real-time, making them suitable for live or ‘as live’ productions.
Dr Florian Block, R&D lead, AI and immersive at dock10 and a reader in digital creativity at the University of York, said: “Lighting is one of the core elements of creative expression in any TV show, but the lighting options for virtual studio productions are currently extremely constrained. That’s what makes this foundational research project so exciting. We are aiming to create a proof of concept that will ultimately lead to the development of a bespoke AI tool for delivering realistic lighting interactions in virtual studio productions in real-time.”
