3D R&D Opinion – The BBC is addressing the challenge of developing production tools that allow 3D productions on television budgets, explains Dr Oliver Grau, lead technologist, BBC Research & Development.
Stereoscopic 3D (S3D) in the cinema and the downscaled version that will soon be introduced into the home can be seen as a step towards the audience’s fuller immersion into programmes. Immersion gives the viewer a new, more intense experience and can be achieved by a number of means: a highly realistic presentation of the real world is the way the cinema industry seems to be heading with S3D.
Another way is through interactivity, where for example the viewer can move through a scene and the ‘virtual world’ around them will react to their presence. The latter functionality has been pioneered by virtual reality (VR) in the 1980s and 90s. Although not that many VR systems seem to have made the way out of the labs to the consumer, one could argue that video games are indeed an implementation of an interactive immersive system.
BBC Research & Development has a long track record of research of ‘3D’ in a wider definition and a number of applications have found their way into practical use over the last years. This includes production techniques and products that allow integration of 3D graphics into real world images in applications as virtual studios or special effects. An example of which would be the virtual characters or ‘Zooks’ featured in BBC Children’s programme Bamzooki.
More recent BBC R&D projects related to 3D content have looked into deriving proper 3D information from multi-camera feeds of the scene (called multi-camera 3D reconstruction). In the ‘iview’ project a system was developed to capture 3D models of sport events like football or rugby. This means that during post-match analysis we can generate new views of incidents from angles that were not covered by any of the match cameras. This feature has obvious applications to enhance sport coverage, but could also be exploited in new areas for example in natural history or factual programmes.
Moreover BBC R&D has looked into how 3D reconstruction can make the production of 3DTV content more cost effective In particular for live sports events like football or rugby, the production of stereo content increases costs significantly. To add S3D to the regular broadcast each camera position would need to be equipped with a pair of cameras in a special stereo rig. That would increase the costs beyond simply doubling the camera budget, since it requires special skills to set-up, to operate and additional broadcast infrastructure, e.g. in OB vans.
Further, special camera systems — like boxed super-zoom-lenses or high-speed cameras that are commonly used for the coverage of sports content — cannot easily be used in a S3D production. Because of their size, for example, they cannot be put into a stereo mirror rig. For this and other reasons, it is very likely that S3D productions will only be able to share some of the conventional broadcast equipment and will be operated mostly alongside the conventional broadcast coverage, which significantly increases costs.
Reconstruction techniques
At IBC 2009 the BBC presented an alternative production method, which derives 3D stereoscopic content from the normal broadcast coverage cameras, plus optional locked-off cameras. The approach makes use of the multi-camera 3D reconstruction techniques I mentioned before. These techniques perform an automatic camera calibration and segmentation of the foreground action. From this information a 3D model of the foreground action is computed. This data is then converted into a 3DTV format.
As part of the EU-funded 3D4You project BBC R&D has investigated 3DTV formats that are independent from the 3D display as it stores depth information alongside the image, allowing the generation of either stereoscopic or multi-view formats for different kinds of 3D displays. This allows us to adjust the 3D content to the display size and viewing distance. This addresses the problem that content is currently shot with a fixed interocular distance (the distance between the capture cameras in a stereo rig) targeted for a cinema screen. This is not necessarily optimal for use in the home on a far smaller screen.
As the last example shows, S3D holds currently a lot of trade-offs in order to present the viewer a more immersive experience. It still remains to be seen whether the increased production cost of S3D is justified in terms of the benefit perceived by the viewer. The biggest challenge for R&D is to develop production tools that allow 3D productions on TV budgets.
As is always the case with technological innovations at the BBC we are committed to exploring 3D developments in television on behalf of licence fee payers to examine how they may add to the delivery of our public service objectives. So this three-dimensional story is not over yet.
