Deep Learning Based Source Separation Applied To Choir Ensembles

Abstract

Choral singing is a widely practiced form of ensemble singing wherein a group of people sing simultaneously in polyphonic harmony. The most commonly practiced setting for choir ensembles consists of four parts; Soprano, Alto, Tenor and Bass (SATB), each with its own range of fundamental frequencies (F0s). The task of source separation for this choral setting entails separating the SATB mixture into the constituent parts. Source separation for musical mixtures is well studied and many deep learning based methodologies have been proposed for the same. However, most of the research has been focused on a typical case which consists in separating vocal, percussion and bass sources from a mixture, each of which has a distinct spectral structure. In contrast, the simultaneous and harmonic nature of ensemble singing leads to high structural similarity and overlap between the spectral components of the sources in a choral mixture, making source separation for choirs a harder task than the typical case. This, along with the lack of an appropriate consolidated dataset has led to a dearth of research in the field so far. In this paper we first assess how well some of the recently developed methodologies for musical source separation perform for the case of SATB choirs. We then propose a novel domain-specific adaptation for conditioning the recently proposed U-Net architecture for musical source separation using the fundamental frequency contour of each of the singing groups and demonstrate that our proposed approach surpasses results from domain-agnostic architectures.

A cappella: Audio-visual Singing Voice Separation

“…The U-Net architecture has been extensively used both in audio-only source separation -both on the spectral [19,20,21] and time [22] domains -as well as in its audio-visual counterpart [23,24,25,26,27,28]. We can also find works on source separation that condition the U-Net on prior information such as the presence of certain types of musical instruments [29], phoneme activation for singing voice separation [30] or the fundamental frequency contour of each type of voice sources in choir ensembles [31].…”