By Stephen Brownsill, Audio Products Manager at TSL Products
Mono, stereo and surround sound: the road to object-based audio
Audio has always played a major part in entertainment media. It has made the big screen in the cinema not only seem bigger but also enveloped the audience with music and sound effects to further wrap them up in the visual story. This approach has also been exploited by television - particularly for sports coverage and video games.
For all three, audio technology has been pushed beyond stereo into the realms of surround sound, predominantly Dolby Digital (5.1 and 7.1). This gave sound designers and dubbing mixers more freedom regarding where specific sounds are placed in a mix - with the ability to have effects in the rear loudspeakers - and how they move - typically panning left or right - in the sonic 'picture.'
While surround sound systems such as Dolby Digital and DTS gave more of an immersive approximation of how humans hear sound in the real world - adding length to the width already offered by stereo and even mono - it was still not a fully realistic soundscape. The limitations are due to these formats being based on a set number of audio channels. Systems such as Hamasaki 22.2 and Auro-3D (11.1) pushed the channel-based concept as far as possible but the growth in 4K/Ultra HD visuals and virtual/augmented reality (VR/AR) has called for audio technology that goes beyond such physical constraints.
This has come in the form of object-based audio (OBA). It offers greater capacity and flexibility to give a higher degree of creative opportunities. It is something that a growing number of customers have been expressing an interest in over the last three years and which needs to be considered seriously, not just in terms of creation and mixing but also how OBA material is monitored.
Next Generation Audio: object-based and spatial sound
OBA is the underlying technology of Next Generation Audio (NGA), the umbrella term for several audio formats that have been developed in recent years for cinema, broadcast and general media. These include Dolby AC-4 and MPEG-H 3D audio, both of which comprise a range of 'tools,' including spatial sound, alternative languages and commentaries, audio description and 'personalisation,' allowing viewers to select settings that match their needs and preferences.
Spatial audio - also known as immersive or 3D audio - is the most recognisable element of NGA. And Dolby Atmos, which is part of AC-4, is the best known and most widely used example of the technology. Like MPEG-H 3D Audio, Atmos has a foundation of channels but the real move on from previous surround systems is the utilisation of 128 objects to enable even more sounds to be represented.
Both Dolby AC-4 and MPEG-H 3D Audio are part of the DVB specification for Ultra HD but OBA has applications in many other areas beyond TV. The BBC has experimented with objects to produce apps that allow listeners to change the sound balance between commentary and crowd noise. VR/AR are more obvious beneficiaries of object-based spatial audio, which complements the immersive nature of the visuals to create a truly enveloping experience.
OBA can adapt to all these scenarios because it not built on a rigid, linear foundation, as is the case with channels. Instead, the constituent parts of a programme - music, dialogue/commentary, sound effects - are converted into a series of media objects, accompanied by metadata describing how everything is to be assembled. All of these elements are broadcast but it is the receiving device - a computer or a TV with the relevant app - that reassembles everything, as dictated by the metadata.
This method also allows the end result to be customised or personalised: for example, creating a ten-minute version of a 30-minute programme. It is this kind of flexibility that enables OBA to offer accessibility, personalisation and interactivity, as well as a fully immersive experience. Beyond that, it is seen as a means of creating a future-proof representation of audio material. Standards have been developed to make OBA production straightforward, including the Audio Definition Model (ADM). This is part of the EBU (European Broadcasting Union) Core XMI schema and describes object-based, channel-based and transformation-based audio content.
The need to monitor OBA - Dolby Atmos primarily
In most cases, when people refer to OBA they are typically talking about spatial audio and - more often than not - Dolby Atmos. Dolby became the de facto standard for channel-based surround sound and looks to have managed the same feat in OBA with Atmos. Outside of cinema, the main area for OBA/Atmos has been in live TV sports production, although it is now an integral part of the Ultra HD package for streaming services such as Netflix and Amazon Prime.
BT Sport broadcast its first Atmos coverage in 2016 and Sky Sports followed, with both now offering it as an option on Premier League matches and other events. Anyone involved in the broadcast production chain - manufacturers, OB truck operators, platform providers, broadcasters - are, or should be, interested in Dolby Atmos. But, in many ways, the industry as a whole is still catching up with OBA. Dolby developed its own controllers and processing tools but now third-party manufacturers are producing alternatives.
There is a critical need to monitor Atmos because of its 128 objects, each of which has an audio level. It is still a very plain and simple job of monitoring audio levels but there is the challenge of visualising the level of so many individual components, which make up an overall level. TSL monitors tend not to be seen in post-production environments; instead, our units are used in OB vehicles and broadcast centre apparatus rooms or studios. Users of our SAM Q SDI monitor can buy one or two optional MADI licenses, which, because that is two MADI inputs (2x64), it is possible to monitor the 128 individual levels in an Atmos or other OBA signal.
We are very much looking at OBA and MADI continues to be part of our development process. When we carry SAM Q forward and produce new versions in the future, it will become more elaborate and most likely have two MADI inputs as standard. What we are seeing now is that peoples’ monitoring requirements in production - and whether Atmos is part of them - change depending on where they are in the chain.
Whatever happened to silent movies? The power of sound
The impact of audio in feature films is widely recognised these days. In the past there were standout films that exploited the almost primal nature of sound to create tension and drama. Just think of Jaws, with the thrashing and screaming of the shark's victims and John Williams’ ominous music. Movies today have even more sophisticated soundtracks and Dolby Atmos, through OBA, is making them ever more layered and sophisticated.
That level of engagement is also key to sports coverage. The return of live sport after the first UK national COVID lockdown was much anticipated and warmly welcomed. But with many games still being played 'behind closed doors,' the atmosphere is completely different due to the lack of crowds in the stadium.
Sound is massively important in broadcasting and, when fans are allowed back to watch football matches, Wimbledon and Formula 1, objects will play an increasingly crucial part in making viewers at home feel as though they are there.