Multichannel Online Dereverberation Based on Spectral Magnitude Inverse Filtering

Li, Xiaofei; Girin, Laurent; Gannot, Sharon; Horaud, Radu

doi:10.1109/taslp.2019.2919183

Cited by 12 publications

(3 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using this model, the narrowband inverse filtering techniques have been developed for speech dereverberation in [32] based on channel equalization, and in [8], [33] based on linear prediction.…”

Section: B Stft Domain Formulationmentioning

confidence: 99%

SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation

Quan,

2024

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

This work proposes a neural network to extensively exploit spatial information for multichannel joint speech separation, denoising and dereverberation, named SpatialNet. In the short-time Fourier transform (STFT) domain, the proposed network performs end-to-end speech enhancement. It is mainly composed of interleaved narrow-band and cross-band blocks to respectively exploit narrow-band and cross-band spatial information. The narrow-band blocks process frequencies independently, and use self-attention mechanism and temporal convolutional layers to respectively perform spatial-feature-based speaker clustering and temporal smoothing/filtering. The crossband blocks process frames independently, and use full-band linear layer and frequency convolutional layers to respectively learn the correlation between all frequencies and adjacent frequencies. Experiments are conducted on various simulated and real datasets, and the results show that 1) the proposed network achieves the state-of-the-art performance on almost all tasks; 2) the proposed network suffers little from the spectral generalization problem; and 3) the proposed network is indeed performing speaker clustering (demonstrated by attention maps).

show abstract

Section: B Stft Domain Formulationmentioning

confidence: 99%

SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation

Quan,

2024

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the case of speech, reverberations result in loss of intelligibility for people who are hard of hearing and non-native listeners in noiseless enclosures. In noisy enclosures, reverberations reduce the intelligibility even for people with normal listening [ 9 ]. Both noise and reverberation play havoc when the generated speech is already less intelligible (e.g.…”

Section: Introductionmentioning

confidence: 99%

Triple-0: Zero-shot denoising and dereverberation on an end-to-end frozen anechoic speech separation network

Gul,

Khan,

Ur-Rehman

2024

PLoS ONE

View full text Add to dashboard Cite

Speech enhancement is crucial both for human and machine listening applications. Over the last decade, the use of deep learning for speech enhancement has resulted in tremendous improvement over the classical signal processing and machine learning methods. However, training a deep neural network is not only time-consuming; it also requires extensive computational resources and a large training dataset. Transfer learning, i.e. using a pretrained network for a new task, comes to the rescue by reducing the amount of training time, computational resources, and the required dataset, but the network still needs to be fine-tuned for the new task. This paper presents a novel method of speech denoising and dereverberation (SD&D) on an end-to-end frozen binaural anechoic speech separation network. The frozen network requires neither any architectural change nor any fine-tuning for the new task, as is usually required for transfer learning. The interaural cues of a source placed inside noisy and echoic surroundings are given as input to this pretrained network to extract the target speech from noise and reverberation. Although the pretrained model used in this paper has never seen noisy reverberant conditions during its training, it performs satisfactorily for zero-shot testing (ZST) under these conditions. It is because the pretrained model used here has been trained on the direct-path interaural cues of an active source and so it can recognize them even in the presence of echoes and noise. ZST on the same dataset on which the pretrained network was trained (homo-corpus) for the unseen class of interference, has shown considerable improvement over the weighted prediction error (WPE) algorithm in terms of four objective speech quality and intelligibility metrics. Also, the proposed model offers similar performance provided by a deep learning SD&D algorithm for this dataset under varying conditions of noise and reverberations. Similarly, ZST on a different dataset has provided an improvement in intelligibility and almost equivalent quality as provided by the WPE algorithm.

show abstract

“…Finally, the STFT coefficients of target speech (the direct-path speech) are estimated from h[f, t, :] by an output linear layer. The narrow-band block processes frequencies independently, and regular SA and time-convolutional feed forward network (T-ConvFFN) are used for modeling the rich temporal-spatial information in one frequency [6,[20][21][22][23][24] in an offline way. The cross-band block processes time frames independently, which will be kept unchanged and not described in this work, please refer to [13] for more details.…”

Section: Introductionmentioning

confidence: 99%

Multichannel Speech Separation with Narrow-band Conformer

Quan¹,

Li²

2022

Interspeech 2022

Self Cite

View full text Add to dashboard Cite

This work proposes a neural network to extensively exploit spatial information for multichannel joint speech separation, denoising and dereverberation, named SpatialNet. In the short-time Fourier transform (STFT) domain, the proposed network performs end-to-end speech enhancement. It is mainly composed of interleaved narrow-band and cross-band blocks to respectively exploit narrow-band and cross-band spatial information. The narrow-band blocks process frequencies independently, and use self-attention mechanism and temporal convolutional layers to respectively perform spatial-feature-based speaker clustering and temporal smoothing/filtering. The crossband blocks processes frames independently, and use full-band linear layer and frequency convolutional layers to respectively learn the correlation between all frequencies and adjacent frequencies. Experiments are conducted on various simulated and real datasets, and the results show that 1) the proposed network achieves the state-of-the-art performance on almost all tasks; 2) the proposed network suffers little from the spectral generalization problem; and 3) the proposed network is indeed performing speaker clustering (demonstrated by attention maps).

show abstract

Multichannel Online Dereverberation Based on Spectral Magnitude Inverse Filtering

Cited by 12 publications

References 49 publications

SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation

SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation

Triple-0: Zero-shot denoising and dereverberation on an end-to-end frozen anechoic speech separation network

Multichannel Speech Separation with Narrow-band Conformer

Contact Info

Product

Resources

About