Clustering of spatial cues by semantic segmentation for anechoic binaural source separation

Integration of deep learning with expectation maximization for spatial cue-based speech separation in reverberant conditions

“…Removing the phase wrap problem by using the top down approach of [2] did not work well in case of U-Net, as it reduces the IPD variance of each source. This variance reduction works well with the expectation maximization (EM) algorithm [2] but not for the convolutional neural network U-Net [11]. The inclusion of IPD cues (whether the values observed from the mixture or those after the phase unwrap by the top down approach) in SONET [11], resulted in decline of its output performance.…”

“…This variance reduction works well with the expectation maximization (EM) algorithm [2] but not for the convolutional neural network U-Net [11]. The inclusion of IPD cues (whether the values observed from the mixture or those after the phase unwrap by the top down approach) in SONET [11], resulted in decline of its output performance. The performance comparison of the two speech separation models, one using the EM algorithm and the other using the SONET-P network for clustering the IPD cues, is given in the 'experiment' section (Section V).…”

“…SONET [11] is a U-Net (deep learning convolutional neural network) based interaural speech separation model, designed for anechoic conditions. Like MESSL, this model is also based on WDO assumption and uses the spatial cues for source separation.…”

“…In SONET, two separate U-Nets (a specialized neural network for semantic segmentation) are trained on the interaural level difference (ILD) and the interaural phase difference (IPD)) spectrograms generated by a single source. These U-Nets are named as SONET-L (SONET trained on ILD cues) and SONET-P (SONET trained on IPD cues) in [11]. After training, these U-Nets are used to predict the class of each time frequency (TF) unit of the interaural spectrogram of an audio mixture.…”

“…Also, as the training phase of deep neural network is lengthy and requires high-end computational resources (graphical processing unit (GPU)), a 'generalized solution' is suggested in [11] as shown in Figure 4 (b), where the network trained for one source position can be used for other sources placed in its neighboring positions due to similarity of interaural cues generated by these sources.…”