Elicitation of Expert Knowledge to Inform Object-Based Audio Rendering to Different Systems

Abstract

Object-based audio presents the opportunity to optimize audio reproduction for different listening scenarios. Vector base amplitude panning (VBAP) is typically used to render objectbased scenes. Optimizing this process based on knowledge of the perception and practices of experts could result in significant improvements to the end user's listening experience. An experiment was conducted to investigate how content creators perceive changes in the perceptual attributes of the same content rendered to systems with different numbers of channels, and to determine what they would do differently to standard VBAP and matrix based downmixes to minimize these changes. Text mining and clustering of the content creators' responses revealed six general mix processes: the spatial spread of individual objects, EQ and processing, reverberation, position, bass, and level. Logistic regression models show the relationships between the mix processes, perceived changes in perceptual attributes, and the rendering method/speaker layout. The relative frequency of use for the different mix processes was found to differ between categories of audio objects suggesting that any downmix rules should be object category specific. These results give insight into how object-based audio can be used to improve listener experience and provide the first template for doing this across different reproduction systems.

Determination and Validation of Mix Parameters for Modifying Envelopment in Object-Based Audio

“…Literature pertaining to envelopment was used to suggest parameters that met the second criterion. An experiment performed by Woodcock et al [35] investigated the parameters that mixing engineers would use to make manual changes to object-based audio that had been rendered to different loudspeaker layouts, as well as selecting the perceptual attributes that had changed as a result of the rendering. The responses were clustered into a set of parameters (spread, EQ and processing, reverb, position, bass, and level) that were all found to show a significant relationship to changing envelopment.…”

A System Architecture for Semantically Informed Rendering of Object-Based Audio

“…Downmixing based on metadata adaptation is particularly useful for systems where VBAP rendering is not possible without introducing virtual loudspeakers into the configuration, such as two-channel stereo. A benefit to this approach over traditional matrix downmixing methods is that it allows downmixing rules to be specified for different categories of audio objects-it has recently been shown that mix engineers apply different processes to different categories of audio objects when downmixing object-based audio to different systems [10].…”

“…Metadata transformations may be derived from psychoacoustics, subjective listening experiments, or expert knowledge. For example, rules for automatic repositioning of sound objects were derived from re-mixing experiments with sound engineers [10] (see Sec. 2.1).…”

“…• Definition of a metadata representation, extending the schema proposed in [3]; • Introduction of a processing framework for metadata adaptations, generalizing the software introduced in [9] to foster the composition of structured, complex adaptation schemes; • Integration of perceptual metering into the system architecture; • Description of an exemplary extensible open-source implementation to illustrate the concepts of the system architecture, but also to enable experimentation and research into the SIR approach; • Discussion of the system architecture's capabilities through a series of application examples drawn from our previous work [10][11][12][13].…”

User-Influenced/Machine-Controlled Playback: The variPlay Music App Format for Interactive Recorded Music

“…Treating the variPlay stems as objects and embedding appropriate metadata would have provided numerous creative mix/mastering opportunities, but this would also have incurred considerable DSP at the point of rendering and so was avoided. This is the approach proposed by Woodcock et al (2018) specifically for content creators to replicate mix characteristics at the point of rendering. Furthermore, it was not possible to implement passive app-native processing on the output buss due to such CPU-loading considerations, especially for compatibility with older phones.…”