A Deep-Learning Based Framework for Source Separation, Analysis, and Synthesis of Choral Ensembles

Abstract

Choral singing in the soprano, alto, tenor and bass (SATB) format is a widely practiced and studied art form with significant cultural importance. Despite the popularity of the choral setting, it has received little attention in the field of Music Information Retrieval. However, the recent publication of high-quality choral singing datasets as well as recent developments in deep learning based methodologies applied to the field of music and speech processing, have opened new avenues for research in this field. In this paper, we use some of the publicly available choral singing datasets to train and evaluate state-of-the-art source separation algorithms from the speech and music domains for the case of choral singing. Furthermore, we evaluate existing monophonic F0 estimators on the separated unison stems and propose an approximation of the perceived F0 of a unison signal. Additionally, we present a set of applications combining the proposed methodologies, including synthesizing a single singer voice from the unison, and transposing and remixing the separated stems into a synthetic multi-singer choral signal. We finally conduct a set of listening tests to perform a perceptual evaluation of the results we obtain with the proposed methodologies.

Unsupervised Music Source Separation Using Differentiable Parametric Source Models

“…Homogeneous audio sources are not easily distinguishable in the time-frequency domain and pose a permutation problem [20], [21]. While permutation-invariant training is used for supervised speech separation [21], [22], methods for musical homogeneous source separation exploit side-information such as F0 estimates [23], [11] or a musical score [24], [9], [10] to guide the separation.…”

“…In this context, a choir is composed of four homogeneous sources: a soprano, alto, tenor, and a bass singer. Petermann et al [23] modified the conditioned U-Net [25] so that the target source can be selected and separated using its F0 information. Results show that this leads to improved objective separation quality compared to using non-informed source-specific models.…”

“…The method proposed in this paper uses F0 information to separate the (possibly homogeneous) sources like the learning free-methods of [9], [11] and the supervised methods of [23], [24]. It provides better performance than learning-free methods and does not require expensive labeled data like supervised methods.…”

“…Furthermore, we train the F0-informed supervised deep learning approach for vocal ensemble separation proposed by Petermann et al [23] on our data. They use a classical U-Net architecture with a control mechanism [25].…”

A Fully Differentiable Model for Unsupervised Singing Voice Separation

“…Our models consistently outperform Petermann et al's[21] U-Net model in the SI-SDR metric. The best results were achieved Evaluation of proposed approaches on CSD (Test dataset) using source separation and pitch accuracy metrics, for the BC1Song (a) and BCBSQ (b) training datasets.…”