Multi-Microphone Complex Spectral Mapping for Speech Dereverberation

Abstract

This study proposes a multi-microphone complex spectral mapping approach for speech dereverberation on a fixed array geometry. In the proposed approach, a deep neural network (DNN) is trained to predict the real and imaginary (RI) components of direct sound from the stacked reverberant (and noisy) RI components of multiple microphones. We also investigate the integration of multi-microphone complex spectral mapping with beamforming and post-filtering. Experimental results on multi-channel speech dereverberation demonstrate the effectiveness of the proposed approach.

Real-Time Speech Enhancement for Mobile Communication Based on Dual-Channel Complex Spectral Mapping

“…With multi-channel complex spectral mapping, the explicit use of these inter-channel features does not produce performance gain, as shown in Table 3. This suggests that inter-channel features can be captured implicitly through a DNN that is trained for multi-channel complex spectral mapping, which is consistent with [16] for speech dereverberation.…”

“…Another useful spatial cue is the inter-channel phase difference (IPD) or inter-channel time difference (ITD), which is highly correlated with the direction of arrival with respect to the microphone array. Both IID and IPD (or ITD) can be implicitly exploited by performing multi-channel complex spectral mapping [16], where the IID and the IPD are encoded in the dual-channel complex spectrogram of the noisy mixture. In contrast to conventional beamforming that typically exploits second-order statistics of multiple channels, such an approach has the potential to extract all effective cues in dual-channel complex-domain inputs through deep learning.…”

“…The symbol pg denotes the number of weights in each weight group g. Here λ1 and λ2 are predefined weighting factors. Hence, the loss function can be written as L = LRI+Mag+RSGL, where LRI+Mag is the loss function developed in [16].…”

Count And Separate: Incorporating Speaker Counting For Continuous Speaker Separation

“…In this context, a block-online approach is proposed in [13]- [15] to address continuous speaker separation, where speech signals from an unknown number of speakers, degraded by environmental noise, room reverberation and a wide range of speaker overlap, arrive as a continuous stream. These studies assume that in each fixed-length short processing block, typically 2.4-second long, there are at most two speakers talking, so that a two-speaker separation model based on for example utterance-wise PIT (uPIT) can be applied in each block for separation.…”

Time-Domain Loss Modulation Based on Overlap Ratio for Monaural Conversational Speaker Separation

“…With fewer parameters, our best model outperforms the conformer in all scenarios for both utterance-wise and continuous evaluation. We should mention that a very recently posted paper [24] reports state-of-the-art results for the LibriCSS evaluation. This study uses complex spectral mapping to train the separation model.…”