Context-Aware Intelligent Mixing Systems

Lefford, Nyssim; Bromham, Gary; Fazekas, György; Moffat, David

doi:10.17743/jaes.2020.0043

Cited by 10 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a broader context of applications of the proposed approach to automation in general, and the plugin in particular, it will be valuable to investigate how our solution fits alongside semantic audio tools that use content analysis to facilitate workflow automation in audio and media production [36,26,27,37,38]. Further investigation into the context surrounding a workflow [2] and its relation to audio that is being processed would also be valuable. The relationships between context, audio features and audio effects may be represented using appropriate ontologies [39,40,41] and used to guide the parameter selection mechanism either through hyper-parameter optimisation or model selection.…”

Section: Discussionmentioning

confidence: 99%

“…r Grounded theory -This approach employs psychoacoustics and perceptual evaluation studies to understand the mixing process and subsequently the intention of the mixing engineer. It is resource intensive and not always reliable since mix engineers do not always follow a set of rules, making it difficult to encompass all variations within a particular framework, while decisions are often influenced by context [2].…”

Section: Related Workmentioning

confidence: 99%

“…Dynamic range compression has the capability to make a sound punchier, richer and more impactful. However, if used without sufficient knowledge or acknowledgement of style or context [2], compressors can also suppress the musical dynamics and destroy the natural texture of the sound.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Intelligent Control Method for the Dynamic Range Compressor: A User Study

Singh¹,

Bromham²,

Di³

et al. 2021

J. Audio Eng. Soc.

Self Cite

View full text Add to dashboard Cite

Music producers and casual users often seek to replicate dynamic range compression used in a particular recording or production context for their own track. However, not knowing the parameter settings used to produce the audio using the effect may become an impediment, especially for beginners or untrained users who may lack critical listening skills. We address this issue by presenting an automatic compressor plugin relying on a neural network to extract relevant features from a reference signal and estimate compression parameters. The plugin automatically adjusts its parameters to match the input signal with a reference audio recording as closely as possible. Quantitative and qualitative usability evaluation of the plugin was conducted with amateur, pro-amateur and professional music producers. The results established acceptance of the core idea behind the proposed control method across these user groups.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Control Method for the Dynamic Range Compressor: A User Study

Singh¹,

Bromham²,

Di³

et al. 2021

J. Audio Eng. Soc.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The purpose of this approach was to relate the associated semantic terms to the audio transformation. This approach was designed to reveal insight into how the audio changes, because semantic terms are all relative to the original audio content and context [40]. The combined three data sets consist of 40,411 audio samples labelled with 6,247 different descriptors, using five different audio effects.…”

Section: Combining Data Setsmentioning

confidence: 99%

Semantic Music Production: A Meta-Study

Moffat¹,

Man²,

Reiss³

2022

J. Audio Eng. Soc.

Self Cite

View full text Add to dashboard Cite

This paper presents a systematic review of semantic music production, including a metaanalysis of three studies into how individuals use words to describe audio effects within music production. Each study followed different methodologies and stimuli. The SAFE project created audio effect plug-ins that allowed users to report suitable words to describe the perceived result. SocialFX crowdsourced a large data set of how non-professionals described the change that resulted from an effect applied to an audio sample. The Mix Evaluation Data Set performed a series of controlled studies in which students used natural language to comment extensively on the content of different mixes of the same groups of songs. The data sets provided 40,411 audio examples and 7,221 unique word descriptors from 1,646 participants. Analysis showed strong correlations between various audio features, effect parameter settings, and semantic descriptors. Meta-analysis not only revealed consistent use of descriptors among the data sets but also showed key differences that likely resulted from the different participant groups and tasks. To the authors' knowledge, this represents the first meta-study and the largest-ever analysis of music production semantics.

show abstract

“…Beyond integrating audio effects as differentiable operators, a central limitation of deep learning systems for automatic audio production lies in the difficulty of sourcing sufficient data for supervised training, requiring both the unprocessed and final produced recordings and/or parameter data. In addition, the subjective and context-dependent nature of the audio production process further complicates the task [18]. While evidence suggests the existence of "best-practices" [19], learning these techniques in a supervised paradigm is challenging.…”

Section: Introductionmentioning

confidence: 99%

Style Transfer of Audio Effects with Differentiable Signal Processing

Steinmetz¹,

Bryan²,

Reiss³

2022

Preprint

View full text Add to dashboard Cite

We present a framework that can impose the audio effects and production style from one recording to another by example with the goal of simplifying the audio production process. We train a deep neural network to analyze an input recording and a style reference recording, and predict the control parameters of audio effects used to render the output. In contrast to past work, we integrate audio effects as differentiable operators in our framework, perform backpropagation through audio effects, and optimize end-to-end using an audio-domain loss. We use a self-supervised training strategy enabling automatic control of audio effects without the use of any labeled or paired training data. We survey a range of existing and new approaches for differentiable signal processing, showing how each can be integrated into our framework while discussing their trade-offs. We evaluate our approach on both speech and music tasks, demonstrating that our approach generalizes both to unseen recordings and even to sample rates different than those seen during training. Our approach produces convincing production style transfer results with the ability to transform input recordings to produced recordings, yielding audio effect control parameters that enable interpretability and user interaction.

show abstract

Context-Aware Intelligent Mixing Systems

Cited by 10 publications

References 20 publications

Intelligent Control Method for the Dynamic Range Compressor: A User Study

Intelligent Control Method for the Dynamic Range Compressor: A User Study

Semantic Music Production: A Meta-Study

Style Transfer of Audio Effects with Differentiable Signal Processing

Contact Info

Product

Resources

About