Score-Informed Source Separation for Musical Audio Recordings: An overview

Abstract

In recent years, source separation has been a central research topic in music signal processing, with applications in stereo-to-surround up-mixing, remixing tools for DJs or producers, instrument-wise equalizing, karaoke systems, and pre-processing in music analysis tasks. Musical sound sources, however, are often strongly correlated in time and frequency, and without additional knowledge about the sources a decomposition of a musical recording is often infeasible. To simplify this complex task, various methods have been proposed in recent years which exploit the availability of a musical score. The additional instrumentation and note information provided by the score guides the separation process, leading to significant improvements in terms of separation quality and robustness. A major challenge in utilizing this rich source of information is to bridge the gap between high-level musical events specified by the score and their corresponding acoustic realizations in an audio recording. In this article, we review recent developments in score-informed source separation and discuss various strategies for integrating the prior knowledge encoded by the score.

On-the-Fly Audio Source Separation—A Novel User-Friendly Framework

“…The separation performance achievable by these techniques is very limited in reverberant environments [5], [6] where the sources' STFT coefficients are quite overlapped. A more recent class of algorithms known as informed source separation [7], [8] utilizes prior information about the sources to guide the separation process, and was shown to be successful in many contexts using different types of prior information. For instance, such information may include musical scores of the corresponding music sources [7], [9], [10] or text of the corresponding speech sources [8].…”

“…A more recent class of algorithms known as informed source separation [7], [8] utilizes prior information about the sources to guide the separation process, and was shown to be successful in many contexts using different types of prior information. For instance, such information may include musical scores of the corresponding music sources [7], [9], [10] or text of the corresponding speech sources [8]. In some approaches this symbolic information is then converted to audio using a MIDI synthesizer for musical scores [9], [10] or a speech synthesizer for text [8].…”

Music Information Retrieval – Soft Computing Versus Statistics

“…Based on the review presented it may be concluded that issues of retrieval and recommendation are interconnected and these two approaches when joint together may make both processes more reliable. Also, new strategies such as for example separating music tracks at the pre-processing phase [3][8] [28] [29] and extending vector of parameters by descriptors related to a given musical instrument components that are characteristic for the specific musical genre to music genre classification should be more thoroughly pursued [3][28] [29].…”

Automatic music genre classification based on musical instrument track separation

“…Among the most promising, one finds sinusoidal modeling (SM) (Serra and Smith 1990) that was extensively exploited over the last two decades. There are also many examples of algorithms that were implemented within many research studies (Bregman 1990;Casey and Westner 2000;de Cheveigne 1993;Dziubiński et al 2005;Eweret et al 2014;Gerber et al 2012;Gillet and Richard 2008;Herrera et al 2000). Uhle et al (2003) designed a system for drum beat separation based on Independent Component Analysis.…”