The team at BBC Research & Development have been working on a project looking at how machine learning can help colourise archive black and white video.
In a blog post, the team reveal they are working on researching how to colourise black and white images using other colour images with similar content to guide the colourisation process.
The post explains that mapping colours from a greyscale input can be a complex task. “For example, a car can be red, blue, or an infinite array of colours, and the algorithm will select the most probable colour, which may not necessarily meet what the user expected to see,” says the post.
This had led the team at BBC R&D to research more conservative solutions and provide colour references to guide the system towards more accurate colour predictions.
They have developed a novel exemplar-based neural network, called XCNET, that achieves fast and high-quality colour predictions. Unlike other processes that involve two steps, XCNET transfers both the style and the colourisation at the same time – simplifying the process and achieving faster predictions.
According to the blog, XCNET is composed of three different branches: the first takes the black & white target (T) and colour reference (R) and outputs volumes of features.
The second branch integrates attention modules, such as vision transformers, to fuse information from both sources of features. The combined features are then transformed into actual colours using the third branch (pyramid decoder).
The aim of the project, says the post, is to provide the most realistic colourisation and restoration of high resolution black and white video.
The full blog post is available to read here.