Online/offline score informed music signal decomposition: application to minus one

Abstract

In this paper, we propose a score-informed source separation framework based on non-negative matrix factorization (NMF) and dynamic time warping (DTW) that suits for both offline and online systems. The proposed framework is composed of three stages: training, alignment, and separation. In the training stage, the score is encoded as a sequence of individual occurrences and unique combinations of notes denoted as score units. Then, we proposed a NMF-based signal model where the basis functions for each score unit are represented as a weighted combination of spectral patterns for each note and instrument in the score obtained from a trained a priori over-completed dictionary. In the alignment stage, the time-varying gains are estimated at frame level by computing the projection of each score unit basis function over the captured audio signal. Then, under the assumption that only a score unit is active at a time, we propose an online DTW scheme to synchronize the score information with the performance. Finally, in the separation stage, the obtained gains are refined using local low-rank NMF and the separated sources are obtained using a soft-filter strategy. The framework has been evaluated and compared with other state-of-the-art methods for single channel source separation of small ensembles and large orchestra ensembles obtaining reliable results in terms of SDR and SIR. Finally, our method has been evaluated in the specific task of acoustic minus one, and some demos are presented. which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.f ,t X(f , t)log X(f ,t) X(f ,t)− X(f , t) +X(f , t) β = 1 f ,t X(f ,t) X(f ,t)+ log X(f ,t) X(f ,t)

Sparse pursuit and dictionary learning for blind source separation in polyphonic music recordings

“…One of the main challenges in score-informed separation is the alignment of the score with the audio recording. For this, a combined approach has recently been proposed by Munoz-Montoro et al [34].…”

Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Sounds

“…This approach can be applied to our problem. The recent literature has shown that the use of musical scores enhances the separation performance [12][13][14][15]. The method using nonnegative matrix factorization (NMF) presented in [15] is one of the state-of-the-art score-informed source separation methods.…”

“…Owing to the interpretability of the synthesis parameters, we can use musical score information for the initialization of the synthesis parameters. Recent source separation literature has shown that the use of score information improves the separation performance [12][13][14][15], which may be true for our problem. We experimentally examine the effect of the score-based initialization of the synthesis parameters.…”

Parallel multichannel music source separation system

“…In this stage, the magnitude spectrograms of each audio channel are estimated using a NMF approach. For this aim, we propose a multichannel extension of the signal model presented in [27]. Muñoz et al proposed a signal model where the score information is encoded in the form of score units, so that each basis represents a unique combination of notes defined in the score.…”