Score-informed audio decomposition and applications

Abstract

The separation of different sound sources from polyphonic music recordings constitutes a complex task since one has to account for different musical and acoustical aspects. In the last years, various score-informed procedures have been suggested where musical cues such as pitch, timing, and track information are used to support the source separation process. In this paper, we discuss a framework for decomposing a given music recording into notewise audio events which serve as elementary building blocks. In particular, we introduce an interface that employs the additional score information to provide a natural way for a user to interact with these audio events. By simply selecting arbitrary note groups within the score a user can access, modify, or analyze corresponding events in a given audio recording. In this way, our framework not only opens up new ways for audio editing applications, but also serves as a valuable tool for evaluating and better understanding the results of source separation algorithms.

Reverse Engineering the Amen Break — Score-Informed Separation and Restoration Applied to Drum Recordings

“…When striving for good perceptual quality of the separated target signals, many authors propose to use priors for the decomposition [21], [13], [14], [22], [23], [8]. This has the advantage that the separation can be guided and constrained by information on the approximate location of component signals in time (onset, offset) and frequency (pitch, timbre).…”

Online score‐informed source separation in polyphonic mixtures using instrument spectral patterns

“…Over the last years, this field has been widely addressed with novel ideas and application scenarios such as instrument‐wise equalization, personal music remixing, music information retrieval, and intelligent audio editing . In the context of classical music, source separation could be applied for different interesting scenarios.…”

“…In this way, a large amount of MIDI scores are available on the web at sources such as classical MIDI connection. Source separation guided by a musical score is called score‐informed source separation . In this scenario, a prior alignment between the score and audio is required.…”

“…Consequently, typical statistical methods such as independent component analysis 2,3 or nonnegative matrix factorization (NMF) 4 often fail to completely recover individual sound objects from music mixtures. 5 Over the last years, this field has been widely addressed with novel ideas and application scenarios 6 such as instrument-wise equalization, 7 personal music remixing, 8 music information retrieval, 9 and intelligent audio editing. 10 In the context of classical music, source separation could be applied for different interesting scenarios.…”

“…‡ Source separation guided by a musical score is called score-informed source separation. 8,9,11 In this scenario, a prior alignment between the score and audio is required.…”