Files are displacing videotape and film in acquisition and delivery. This raises the question ‘if we begin with files, post produce files and deliver files, why do we need conventional post and broadcast products? Can’t commodity IT kit do the job?’
The need has never been greater to get costs under control. Increasingly, commodity IT products can do the job — but there’s growing evidence that to really drive down costs, we have to stop simply dropping IT-based products into conventional post and broadcast architectures and really start to think about business processes. That’s why Service Oriented Architecture (SOA) is now a hot topic. SOA is not a technology but rather a design philosophy widely used outside our industry, write Bruce Devlin and Mark Horton of AmberFin.
Before looking at SOA in detail, we need to understand the problems with traditional thinking. In the past vendors built hardware products to sell into traditional videotape and film workflows. The products used traditional control methods to integrate with other vendors – like MOS, VDCP, RS-422, GPIs and proprietary APIs. It all worked, in a traditional way.
Then, as computers and disk storage grew in capability, vendors built products that included computers and disk storage – but were naturally careful to continue to support traditional workflows and use traditional control and connection protocols, like MOS, VDCP, RS-422, GPIs and proprietary APIs.
Today, many vendors use commodity computers and storage as product building blocks – or mix computers plus hardware – but the computers are often ‘locked into’ the product. Vendors don’t always offer broadcasters, post houses or integrators the choice of simply providing the software and allowing the user to buy the computers and storage themselves.
Partly because of this ‘lock in’, the computers can’t talk to other computers in a computer-friendly way. The way they are architected means they have to use traditional control and connection protocols, like MOS, VDCP, RS-422, GPIs and proprietary APIs. This can create tricky interdependencies between different vendor applications that get can broken when it is time to change.
The growth of IT
Editing, compositing, graphic design and CGI are now dominated by commodity IT products. More recently, content conversion and mastering and Digital Intermediate have seen a sharp growth in use of commodity IT products. Even the new and data-intensive s3D business has a range of competent and cost-effective commodity IT-based products.
There are still some specific areas where traditional products can offer advantages — for now. For example live news, sports or events production, where on-the-fly baseband video still dominates, shows no sign of changing but the overall industry trend is towards commodity IT.
However there is a serious point that current integration and interoperability problems create costs. But the issues in integration and data management are often not caused by the commodity IT products themselves. In many cases they are caused by the architectures in which they are deployed.
It’s no surprise to most that our industry is now crossing over to commodity IT products. This process has been happening for years. In the past, similar changes happened in related industries like print and audio. What is more controversial is how IT has been deployed. Many now argue that we are forcing IT into inefficient traditional architectures, rather than taking the opportunity to rethink our business processes. These traditional architectures can make scaling or upgrading very expensive – in some cases it simply isn’t possible to efficiently integrate traditional products into new workflows.
Enter SOA
SOA is an IT design philosophy, not a technology. A key idea in SOA is to use ‘loose’ rather than ‘tight’ coupling to integrate applications. ‘Loose coupling’ means applications have little dependence on each other. The impacts of change are minimised — there is little or no dependency on the integration of product X with product Y. This is a good thing.
‘Tight coupling’ means products are integrated with each other in specific and potentially inflexible way. Any change to the business may prove to have expensive repercussions in terms of re-engineering. Someone has to pay for that, one way or another. A bad thing.
A very simple example of tight coupling in our industry is the RS422 interface for VTR or VDCP control. At a business level, we know that the 9pm movie is going to start exactly at 9pm. We know days in advance what the schedule is going to be and for most of the stations in the world, most of the time, we could theoretically tell the server 24 hours in advance when to start playing. In practice, we still use old RS422 protocols where the server is cued and then given up to 40ms notice that it should start playing.
The play command is tightly coupled to the physical wiring of the RS422 cable. It is tightly coupled in time to the frame before we need to start. It is tightly coupled to the specific serial protocol on the RS422 cable. It is also tightly coupled to the specification of time (i.e. timecode) that the industry uses and mis-uses on a daily basis.
In this example, changing the system to be loosely coupled requires significant changes to the way we think about the ‘play’ command. SMPTE and the EBU are working on standards to help convey a time label that allows more loosely coupled environments to be built.
The tightly coupled playout environment evolved because broadcasting has been battling the boundaries of semiconductor physics for a couple of decades. Now that we can do many functions in software, we are able to draw a box around the playout function and ask ourselves ‘What service does the playout function provide?’
SOA thinking
The current excitement around SOA and loose coupling is that it helps us consider the services that our traditional engineering functions provide for the underlying media business. For example, the service that a transcoder provides is to convert files that you have into files that you need. This has interesting repercussions such as:
- If the file that you have is already in the format that you need, then the Transcoder doesn’t have to do anything. But the notion of it passing through a ‘null’ Transcoder helps you to identify the service boundaries in a workflow.
- A Transcode could be the same as an FTP transfer. This is a good thing. If the business process doesn’t actually have to do anything a lot of the time, then it is low cost and efficient.
- Converting the file that you have into the file that you need might involve a rewrap or decode then re-encode with a different codec. This is the traditional engineering definition of the word ‘Transcode’.
- If the file that you have is the wrong aspect ratio, resolution, frame rate, or is interlaced, then the Transcode could become an aspect ratio conversion, an up/down conversion, a deinterlacer or a full motion–compensated standards conversion.
From a business process point of view, all four of these Transcodes represent the same business process that can be implemented with the same interface. What changes between these functions is the CPU/memory/disk/network loading. Viewing them all as the same process allows significant simplifications in the workflow’s design.
Sceptics at this point might so ‘OK but that’s just Transcoding’. This underestimates how powerful modern Transcoding products have become. Nearly all the video processing functions that were traditionally called “conversion products” are now just a subset of Transcode functionality. Transcoder capabilities can now also include automated ingest of tape or files, quality control, format conversion, standards conversion, cut and splice editing, logo insertion, text insertion, closed captioning handling, image reframing, image resizing, image cropping, audio channel mapping, audio/video sync adjustment and content watermarking.
SOA is also potentially applicable to a wide range of other applications – billing, playout, media movement etc. where there are ‘baseline’ similarities that cross vendor boundaries. We will get the benefits of scale if we can define a set of services that are common to several broadcasters, content owners and other media companies. These common services, when accessed with standardised service interfaces, herald the possibility of ‘plug & play’ in the broadcast infrastructure.
SOA deployment – FIMS
The EBU and AMWA are currently developing a Framework for Interoperable Media Services (FIMS), an important step towards SOA deployment. This has attracted growing support from end users and vendors.
The goal of FIMS is very simple — define what services are required to create a set of ‘software building blocks’ from which the industry can construct the range of facilities and infrastructures needed to meet the demands of the new file based world.
The project is concentrating on both the ‘Framework’ in which the services must operate and a number of key individual services that must operate within the framework. The project is open to any person or company that signs the participation agreement.
One area in which AmberFin has made a contribution is the definition of an abstract Transcode interface. The implications of an agreed interface at this level are quite profound. To restate the earlier point, a modern software Transcoder often contains a vast toolset. That toolset corresponds to a range of single function conventional products. Modern Transcoders can effectively perform most non-creative tasks needed for many kinds of content production.
The open API proposed by AmberFin ensures loose coupling and vendor independence. The API can be used on a job by job basis to make similar profiles by making defined vendor-independent changes to that profile. This allows vendors to differentiate themselves based on speed, picture quality, throughput or whatever else makes a difference to the product. This also allows users to take advantage of any Transcoder that implements the API without needing to implement custom code.
The bottom line
Traditional Broadcast architectures based on traditional broadcast products can work perfectly well when they are first installed but can become very complicated and expensive to scale or update. We believe media organisations looking to create enterprise-level new systems (or even make smaller scale investments) can use SOA thinking to design cost effective and agile workflows.
SOA is well established in other industries and could potentially make a valuable contribution to ours. Vendors want customers to buy their product and if there is sufficient end-user demand, SOA could become widespread.
To find out more about SOA and FIMS go to
www.amwa.tv/projects/MSAG_projects.shtml
Files are displacing videotape and film in acquisition and delivery. This raises the question ‘if we begin with files, post produce files and deliver files, why do we need conventional post and broadcast products? Can’t commodity IT kit do the job?’
The need has never been greater to get costs under control. Increasingly, commodity IT products can do the job — but there’s growing evidence that to really drive down costs, we have to stop simply dropping IT-based products into conventional post and broadcast architectures and really start to think about business processes. That’s why Service Oriented Architecture (SOA) is now a hot topic. SOA is not a technology but rather a design philosophy widely used outside our industry, write Bruce Devlin and Mark Horton of AmberFin.
Before looking at SOA in detail, we need to understand the problems with traditional thinking. In the past vendors built hardware products to sell into traditional videotape and film workflows. The products used traditional control methods to integrate with other vendors – like MOS, VDCP, RS-422, GPIs and proprietary APIs. It all worked, in a traditional way.
Then, as computers and disk storage grew in capability, vendors built products that included computers and disk storage – but were naturally careful to continue to support traditional workflows and use traditional control and connection protocols, like MOS, VDCP, RS-422, GPIs and proprietary APIs.
Today, many vendors use commodity computers and storage as product building blocks – or mix computers plus hardware – but the computers are often ‘locked into’ the product. Vendors don’t always offer broadcasters, post houses or integrators the choice of simply providing the software and allowing the user to buy the computers and storage themselves.
Partly because of this ‘lock in’, the computers can’t talk to other computers in a computer-friendly way. The way they are architected means they have to use traditional control and connection protocols, like MOS, VDCP, RS-422, GPIs and proprietary APIs. This can create tricky interdependencies between different vendor applications that get can broken when it is time to change.
The growth of IT
Editing, compositing, graphic design and CGI are now dominated by commodity IT products. More recently, content conversion and mastering and Digital Intermediate have seen a sharp growth in use of commodity IT products. Even the new and data-intensive s3D business has a range of competent and cost-effective commodity IT-based products.
There are still some specific areas where traditional products can offer advantages — for now. For example live news, sports or events production, where on-the-fly baseband video still dominates, shows no sign of changing but the overall industry trend is towards commodity IT.
However there is a serious point that current integration and interoperability problems create costs. But the issues in integration and data management are often not caused by the commodity IT products themselves. In many cases they are caused by the architectures in which they are deployed.
It’s no surprise to most that our industry is now crossing over to commodity IT products. This process has been happening for years. In the past, similar changes happened in related industries like print and audio. What is more controversial is how IT has been deployed. Many now argue that we are forcing IT into inefficient traditional architectures, rather than taking the opportunity to rethink our business processes. These traditional architectures can make scaling or upgrading very expensive – in some cases it simply isn’t possible to efficiently integrate traditional products into new workflows.
Enter SOA
SOA is an IT design philosophy, not a technology. A key idea in SOA is to use ‘loose’ rather than ‘tight’ coupling to integrate applications. ‘Loose coupling’ means applications have little dependence on each other. The impacts of change are minimised — there is little or no dependency on the integration of product X with product Y. This is a good thing.
‘Tight coupling’ means products are integrated with each other in specific and potentially inflexible way. Any change to the business may prove to have expensive repercussions in terms of re-engineering. Someone has to pay for that, one way or another. A bad thing.
A very simple example of tight coupling in our industry is the RS422 interface for VTR or VDCP control. At a business level, we know that the 9pm movie is going to start exactly at 9pm. We know days in advance what the schedule is going to be and for most of the stations in the world, most of the time, we could theoretically tell the server 24 hours in advance when to start playing. In practice, we still use old RS422 protocols where the server is cued and then given up to 40ms notice that it should start playing.
The play command is tightly coupled to the physical wiring of the RS422 cable. It is tightly coupled in time to the frame before we need to start. It is tightly coupled to the specific serial protocol on the RS422 cable. It is also tightly coupled to the specification of time (i.e. timecode) that the industry uses and mis-uses on a daily basis.
In this example, changing the system to be loosely coupled requires significant changes to the way we think about the ‘play’ command. SMPTE and the EBU are working on standards to help convey a time label that allows more loosely coupled environments to be built.
The tightly coupled playout environment evolved because broadcasting has been battling the boundaries of semiconductor physics for a couple of decades. Now that we can do many functions in software, we are able to draw a box around the playout function and ask ourselves ‘What service does the playout function provide?’
SOA thinking
The current excitement around SOA and loose coupling is that it helps us consider the services that our traditional engineering functions provide for the underlying media business. For example, the service that a transcoder provides is to convert files that you have into files that you need. This has interesting repercussions such as:
- If the file that you have is already in the format that you need, then the Transcoder doesn’t have to do anything. But the notion of it passing through a ‘null’ Transcoder helps you to identify the service boundaries in a workflow.
- A Transcode could be the same as an FTP transfer. This is a good thing. If the business process doesn’t actually have to do anything a lot of the time, then it is low cost and efficient.
- Converting the file that you have into the file that you need might involve a rewrap or decode then re-encode with a different codec. This is the traditional engineering definition of the word ‘Transcode’.
- If the file that you have is the wrong aspect ratio, resolution, frame rate, or is interlaced, then the Transcode could become an aspect ratio conversion, an up/down conversion, a deinterlacer or a full motion–compensated standards conversion.
From a business process point of view, all four of these Transcodes represent the same business process that can be implemented with the same interface. What changes between these functions is the CPU/memory/disk/network loading. Viewing them all as the same process allows significant simplifications in the workflow’s design.
Sceptics at this point might so ‘OK but that’s just Transcoding’. This underestimates how powerful modern Transcoding products have become. Nearly all the video processing functions that were traditionally called “conversion products” are now just a subset of Transcode functionality. Transcoder capabilities can now also include automated ingest of tape or files, quality control, format conversion, standards conversion, cut and splice editing, logo insertion, text insertion, closed captioning handling, image reframing, image resizing, image cropping, audio channel mapping, audio/video sync adjustment and content watermarking.
SOA is also potentially applicable to a wide range of other applications – billing, playout, media movement etc. where there are ‘baseline’ similarities that cross vendor boundaries. We will get the benefits of scale if we can define a set of services that are common to several broadcasters, content owners and other media companies. These common services, when accessed with standardised service interfaces, herald the possibility of ‘plug & play’ in the broadcast infrastructure.
SOA deployment – FIMS
The EBU and AMWA are currently developing a Framework for Interoperable Media Services (FIMS), an important step towards SOA deployment. This has attracted growing support from end users and vendors.
The goal of FIMS is very simple — define what services are required to create a set of ‘software building blocks’ from which the industry can construct the range of facilities and infrastructures needed to meet the demands of the new file based world.
The project is concentrating on both the ‘Framework’ in which the services must operate and a number of key individual services that must operate within the framework. The project is open to any person or company that signs the participation agreement.
One area in which AmberFin has made a contribution is the definition of an abstract Transcode interface. The implications of an agreed interface at this level are quite profound. To restate the earlier point, a modern software Transcoder often contains a vast toolset. That toolset corresponds to a range of single function conventional products. Modern Transcoders can effectively perform most non-creative tasks needed for many kinds of content production.
The open API proposed by AmberFin ensures loose coupling and vendor independence. The API can be used on a job by job basis to make similar profiles by making defined vendor-independent changes to that profile. This allows vendors to differentiate themselves based on speed, picture quality, throughput or whatever else makes a difference to the product. This also allows users to take advantage of any Transcoder that implements the API without needing to implement custom code.
The bottom line
Traditional Broadcast architectures based on traditional broadcast products can work perfectly well when they are first installed but can become very complicated and expensive to scale or update. We believe media organisations looking to create enterprise-level new systems (or even make smaller scale investments) can use SOA thinking to design cost effective and agile workflows.
SOA is well established in other industries and could potentially make a valuable contribution to ours. Vendors want customers to buy their product and if there is sufficient end-user demand, SOA could become widespread.
To find out more about SOA and FIMS go to
www.amwa.tv/projects/MSAG_projects.shtml
