Voice Assignment in Vocal Quartets Using Deep Learning Models Based on Pitch Salience

Abstract

This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios.

Unsupervised Music Source Separation Using Differentiable Parametric Source Models

“…We assume that the fundamental frequencies for each of the J sources can be obtained from the mixture signal with a multiple F0 estimation system. Given that many such systems exist [45], [18], [46] and that it is still an active research area, we are confident that this is a reasonable assumption. When all F0s are obtained, each F0 value needs to be assigned to one specific source.…”

“…Section IV-B. F0 estimates are usually provided at a frame rate which is smaller than the sample rate [45], [18], [46]. Therefore, following [17], the source specific F0 time series are upsampled to the sample rate using bilinear interpolation.…”

“…The F0s are obtained from the mixture signals using the multiple F0 estimation model of Cuesta et al [18]. We use the pre-trained "Model 3" which is available online 6 .…”

“…In [11], H F 0 is initialized using F0 information of the predominant source estimated using the signal model in (16). We initialize H F 0 using the F0 information we obtained from the multi-pitch estimation [18]. In [11], the spectral templates of the residual sources W O ∈ R F ×R and their activations H O ∈ R R×N are initialized randomly.…”

“…Separation is achieved because the F0s for all sources are estimated from the mixture and assigned to the sources beforehand. This can be done using existing methods such as [18], [19].…”