The BBC’s research and development team has worked alongside BBC Wales to produce an immersive audio version of a Six Nations rugby match.
The experience is based on the Scrum V highlights package of the Scotland v Wales Six Nations match on Saturday 13th February 2021.
The listener can watch the video and hear crowd noise and commentary on their main device. But they can also connect extra devices to hear additional crowd noise, giving them an immersive experience and the ability to add sound effects track consisting of the referee’s microphone and pitch sounds (e.g. ball kicks and players shouting).
In a blog post, Jon Francombe from BBC R&D explained that the team faced a number of technical challenges. As well as creating the content, they needed to be able to send different parts of the audio to different connected devices which means they had to have access to the separate audio tracks, not just the complete mix.
“We got hold of the highlights video with the commentary separated from the crowd noise but only had access to the referee and sound effects as a separate recording from the whole match – so it was necessary to find and extract the highlight clips,” explains Francombe. “This was achieved with a process that was part-manual, part-automated. The approximate time of the highlight in the full match was found by hand, then a signal processing technique called cross-correlation (which works by determining the similarity between two audio signals) was used to find the exact clip.”
With the content in place, the team needed to expand its orchestration framework to replay video. “To do this, we modified the BBC Standard Media Player plugin that was used to drive the synchronisation in BBC Together (our experimental tool for remote shared viewing) so that we could hook it up to the audio device orchestration template,” explains Francombe. “We also made some changes to the user experience, for example, providing a prompt to let viewers know that they can connect extra devices, a button to show the connection instructions, and a switch for turning the commentary on and off.”
The audio is now available for rugby, and other, fans to trial via the BBC’s Taster website.