We have all seen worrying headlines over the past year that have underlined the dangers of failing to put in place adequate video playout Disaster Recovery (DR) protection. The fact is, ensuring that broadcast and linear streaming playout is protected should the worst occur is no simple matter – nor is finding a cost-effective way of doing so. Yet it is absolutely essential.
It has become all too clear that disasters such as fire, flood, or the accidental cutting of power or critical cabling can and do cause outages. The potential impact of such events emerged as a priority for many media company executives after multiple UK channels suffered major disruptions in playout services due to a fire-related incident at a central data centre in October 2021 – with downtime ranging from a few minutes to hours. It is a concern that many will still be grappling with as we move into 2023.
Even near-perfect 99.99 per cent (“four nines”) playout reliability means a statistical average of about an hour of downtime a year. The financial and reputational impacts of downtime on media organisations have grown exponentially, along with the complexity of playout and the growth of linked services such as over-the-top (OTT) and international distribution. There is increasingly a knock-on effect that amplifies the risks associated with losing playout – even for just a short time.
While the biggest and most well-funded broadcasters can typically run a full active/active on-premise DR set-up – a configuration in which the playout infrastructure is replicated in another location so that workloads run either concurrently or with almost instantaneous workload switch-over if you lose one site. But the costs are high and often completely out of reach for many smaller and more specialised broadcasters and streaming services.
The good news is that the virtualisation of playout is opening a way for the media sector to replace the monolithic approach to DR that favours only the largest industry players with a lighter, more agile model that leverages cloud capabilities. Advanced cloud playout continues to play a growing role in the everyday operations of media companies of all sizes and descriptions as they deploy cloud-based technologies that give them more scalability, flexibility and efficiency. So, it is natural that the cloud also enables a new DR model that offers all those benefits.
With a cloud-based DR solution, the result is that broadcasters and streaming services can, with the right technology and service partner, implement a broadcast-grade DR solution with minimal recurring costs until the service is activated. This “pay-as-you-use” cloud DR approach means there is a minimal running cost when the backup system is not in playout mode. The full cost of cloud playout only kicks in when there is an outage – although the system must be fired up and tested monthly to ensure it delivers when needed. Often DR services aren’t tested regularly, so when needed, they sometimes don’t work as expected – if at all – but testing and then shifting back into standby mode is easy with a cloud-based solution. In addition, this cloud-based approach enables media companies to scale up DR as they grow through the addition of new channels and services that also need to be protected.
Critically, a cloud DR programme needs to be separate, secure and independent – whether it is to protect an onsite playout environment or supplement an existing managed playout service. That way, it will keep broadcasters on-air if the worst-case scenario should occur. To ensure cloud DR system requires no changes to the existing playout systems or workflow, it should be integrated as part of an ‘out-of-line’ deployment. This approach is crucial to streamlining deployment and means that the backup is truly independent of the existing playout chain – making the backup feed more resilient to related failures.
It is worth noting that good disaster recovery is not simply about moving to the cloud. Implementing a solution that is also facility- and location-independent is fundamental to minimising risk. If the backup is essentially in a building in the same local region as the primary facility, then it could be exposed to the same geographic risks knocking out the main playout – say, a blackout or massive storm hitting the area. Many major international broadcast players benefit from a multi-country architecture, combining on-premise infrastructure with backup disaster recovery in a cloud facility based in a different region or country to minimise risk.
Increasingly, companies are adopting a hybrid playout infrastructure that mixes on-premise and cloud elements. Not everyone wants to move their whole playout load onto the cloud. Many broadcasters maintain their core channels and high-value content, such as premier sporting events, through on-premise infrastructure, due to heavy graphics, live ingest and processing requirements. A cloud-based DR solution can back up all playout models – or be part of a combination of on-premise and virtualised protection.
Implementing a DR set-up independent of primary playout requires an experienced technology partner that can advise and work with a media company to provide the best solution for their business both now and in the future across – regardless of their current workflow architecture. The DR provider must have a deep systems integration knowledge and a comprehensive product/solution set (acquisition, media asset management, playout and delivery) capable of working efficiently on cloud or hybrid environments.
Lastly, media brands should see the adoption of cloud-based DR as part of a wider journey to the cloud. The overall flexibility and scalability of cloud-based playout not only helps enable cost-effective DR and the launch of pop-up channels and alternative services, but it improves overall business agility.
