The Ultra HD debate revolves around getting a richer picture to the living room with implications for capture, post production and retail. Is there a need for an enhanced, richer-pixeled version of Ultra HD that includes greater luminance and colour alongside higher frame rate and resolution? Adrian Pennington reports.
Tom Morrod, senior director, IHS Technology at analysts Screen IHS says yes, arguing that higher colour data has a more meaningful impact on the viewing experience than more pixels.
“High Dynamic Range (HDR) is not an increase in transmitted colour space, but it adds colour/contrast estimates to fill the gap between the colour space that the TV can display, and the colour data from the delivered stream,” he says.
TV displays have been performing better than the signal they are fed for a while now. 8-bit depth per pixel has long been the standard for HD video, yet when today's content is acquired at 12-bit, the signal is insufficient to store the extra information.
“When you acquire content with much higher bit depth its quality is dumbed down on the display to 8-bit,” explains Roland Vlaicu, Dolby's senior director of Broadcast Imaging. “If we were to change the bit depth of every pixel today to 12-bit, it would not be compatible with any existing display system. Our proposal is to maintain a higher bit depth for every pixel up to 12-bit and 16-bit - or one day even higher - from the moment of capture to the home. We need to modify the signal to make it compatible with today's infrastructure.”
Dolby Vision is a suite of tools applied to the colour grading process in post and to the encoding of the signal for HEVC decoders in new Ultra HD TVs. Under licence, CE manufacturers will need to integrate Dolby Vision decoding and display mapping software into their equipment. Sets from Sharp, Vizio and TCL will have that capability this year. How Ultra HD will be presented at retail is up for debate, with the Digital TV Group in the UK hoping to avoid the confusion that surrounded the badging of hardware with the HD-ready logo. Dolby says it expects to place a separate Dolby Vision logo on consumer equipment (and possibly on content augmented with its system) alongside a 4K/UHD/Ultra HD logo “as a clear indicator of performance and quality.
“We believe Dolby Vision is the perfect compliment to UHD,” says Vlaicu. “The combination of higher frame rate and higher resolution creates a true-to-life viewing experience.”
Display brightness is measured in nits and the Dolby Vision system can describe a peak of 10,000nits. That's way more capacity than the average 400nits most TVs can display, but Dolby believes that by allowing more information to flow through the chain the better the final picture quality. Dolby has built prototype 4000nit professional reference monitors and is looking to licence the technology to begin to populate facilities. A first set of plug-ins can be expected at NAB for colour grading systems from Filmlight and Blackmagic.
The process is already in action on 25 legacy and in production titles. “New movies are our goal, which will then supply to OTT aggregators to stream,” says Vlaicu. “We are enabling post facilities, primarily in Hollywood, to create Dolby Vision grades with the necessary plug-ins in for colour grading software.”
“The post workflow doesn't need modifying,” says Vlaicu. “Today in a lot of cases, the DP and colourist will reference grade to [Dolby's] PRM 200 monitor in Rec 709 for 100nit displays. We'd prefer that they look at the new Dolby Vision reference display and make creative decisions that enable them to create a Dolby Vision master.
“The process is largely automatic and built into the capability of the plug-ins. With a single pass master versions can be derived for Dolby Vision, rec 709 and DCI-compliance. If the director/DP wants to fine-tune they can make manual decision with the tools provided,” says Vlaicu.
Here's Morrod's verdict: “Extra colour is a serious improvement, but while the Dolby technology makes things ‘pop’ and look more real, the major benefit of additional colour information at transmission is the subtle shading around the edge of things - the slow and accurate transition of bright to dark - which makes things ‘look’ 3D. It’s one of the main visual cues to 3D objects in space, shading. This can’t be ‘created’ by HDR very well, but is what makes 4K demos ‘look 3D’ – they are using additional colour subsampling to place a colour within the display-able colour-space, making it look more realistic.”
Technicolor also bets on brightness
Technicolor is working on a similar set of HDR tools and workflows and will bring two versions of its dynamic range expansion to market. The first is for high-quality legacy content from studios and will be a mix of automatic processes and artistic interactions. Initially these will feed the library of M-Go, the movie rental streaming service it runs with DreamWorks and which will debut a 4K option shortly.
“These projects will be individually regraded by a colourist often with involvement of the DP,” explains Cristina Gomila, director of Technicolor's Rennes Research and Innovation Centre. “Inverse tone mapping allows us to stretch the dynamic range of all legacy catalogue and make the images brighter for new displays. We'll try to stretch the range in a way that conveys the true image the DP wanted to create but in such a way we don't want flush the user with maximum peak luminance at any time.
“Where content or areas of content have been clipped when represented in LDR Technicolor has some algorithms that can compensate and bring detail from brighter zones for example. “Lost information is difficult to recover so we are going to be seeing a variety of quality depending on these algorithms,” says Gomila.
Technicolor will also licence a version of its technology to perform dynamic range expansion at the receiver, in set top boxes or the TV. As with Dolby Vision, the enhancement can be applied to images of any resolution. She calls for a consistent measure of what constitutes HDR to be agreed and is contributing to standardization activities on HEVC extensions for HDR and wide Colour Gamut in MPEG, ITU-T and applications standard committees such as DVB and ATSC 3.0.
“The industry, including standards bodies like MPEG, need to set up a continuous scale and define a point below which we can term LDR (low dynamic range) and above which is HDR,” says Gomila. “There may even be intermediate range, but there should be a single agreed definition to avoid market confusion.”
She suggests that from capture to grade creatives need to learn about how new artistic effects can be conveyed with HDR images. “If you knew you could display this greater range, the question is would you play differently with the lighting when filming? This has implications for productions being shot going forward.”
The genesis of Dolby's technology came from IP acquired from Brightstar in 2007. The Canadian wireless technology specialist had developed the concept of local dimming in which TVs are dynamically lit by individual LEDs arrayed behind the screen, rather than by way of a single backlight. Brightstar developed a means of creating an HDR version of regular content by extrapolating information needed to create the locally dimmed image within the TV. The idea didn't take off because it required TV sets to become wider to accommodate the LEDs at a time when the trend is toward ever thinner units (although LED edge-lit displays which light up the panel equally are now common).
“When we acquired Brightside, the existing IP revolved mostly around the concept of direct LED local dimming for LCD displays,” explains Vlaicu. “The idea of enhancing the signal itself was really just an idea at the time. Since then, we have built the technology and IP to turn this into a reality. The way we see this panning out now is that the combination of better performing displays in the form of local dimming LCD or OLED along with Dolby Vision signals that carry higher dynamic range and a wider colour gamut will enable a far superior imaging performance.”
Technicolor and Dolby are not alone in developing this area. Video processing company Folded Space, subsidiary to anamorphic lens maker Panamorph, is marketing what it terms deep colour content encoding.
“DCE is an extremely efficient process requiring very little additional bandwidth or processing power to deliver true 12-bit equivalent colour to compatible displays,” notes business development director John Schuermann.
The company's algorithms process original content with 12-bits and encode information about the colour detail into a backward compatible 8-bit Blu-ray disc. Newer displays and Blu-ray players with the decoding algorithm can then restore a 12-bit equivalent of the original image. Folded Space is licensing the technology to software partners for free “to stimulate HDR content production” it says, while charging a modest fee for it to player and display vendors.
“We are working with several CE manufacturers and major film studios to develop a 12-to-8-back-to12-bit demo, but of course the current limitation is the lack of a high bit depth display to show it on or even good 12-bit video content to run through the process,” says Schuermann. “These are challenges we are working on with several CE companies and film studios, most under NDA at this point.”
He adds, “The whole HDR/high colour bit depth discussion is evolving rapidly and there are still technical problems to be overcome on the display side. This is where Dolby is focusing part of their efforts, as a major component of what they are doing is encouraging the development of higher brightness, high bit depth displays. We have simply been focusing on the efficient delivery of high bitrate content.
“It is our understanding that Dolby's delivery system will be proprietary and include at least two data streams that will need to be remuxed at the display. Our technology is fully compatible with current and future Blu-ray standards and can be adapted to other high-quality delivery methods such as downloads (although probably not realtime streaming, as the massive compression could adversely affect our process).”