Deep Learning Based Dereverberation of Temporal Envelopes for Robust Speech Recognition

Purushothaman, Anurenjan; Sreeram, Anirudh; Kumar, Rohit; Ganapathy, Sriram

doi:10.21437/interspeech.2020-2283

Cited by 6 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In these experiments, we perform a separate dereverberation and speech recognition E2E In the model defined as lin-ear+transformer, we use the same transformer configuration, but use a simple linear layer to project the feature matrix, which is passed through the transformer. Further, the architecture with 4 CNN and 2 LSTM layers gave the best performance (similar to the previous findings on hybrid ASR model [10]).…”

Section: Reverb Challenge Asrsupporting

confidence: 88%

“…Recently, end-to-end models with attention based modeling have also been explored on the REVERB challenge dataset [21,22]. Previously, we had proposed a convolutional neural network model to perform dereverberation of speech [10,11]. In the current work, we extend this prior work for E2E transformer based ASR system.…”

Section: Literature Reviewmentioning

confidence: 93%

“…The Let xq(n), hq(n) and rq(n) denote the sub-band clean speech, room-response and the reverberant speech respectively for the q th sub-band. The sub-band envelopes of far-field speech mrq(n), extracted using frequency domain linear prediction (FDLP) [23,1], can be approximated as, [10]…”

Section: Signal Modelmentioning

confidence: 99%

“…Our previous work [10,11] explored the use of dereverberation of sub-band envelopes for hybrid speech recognition systems. The sub-band envelopes are extracted using the autoregressive modeling framework of frequency domain linear prediction [12,13].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

End-To-End Speech Recognition with Joint Dereverberation of Sub-Band Autoregressive Envelopes

Kumar

Purushothaman

Sreeram

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

The end-to-end (E2E) automatic speech recognition (ASR) systems are often required to operate in reverberant conditions, where the long-term sub-band envelopes of the speech are temporally smeared. In this paper, we develop a feature enhancement approach using a neural model operating on sub-band temporal envelopes. The temporal envelopes are modeled using the framework of frequency domain linear prediction (FDLP). The neural enhancement model proposed in this paper performs an envelope gain based enhancement of temporal envelopes. The model architecture consists of a combination of convolutional and long short term memory (LSTM) neural network layers. Further, the envelope dereverberation, feature extraction and acoustic modeling using transformer based E2E ASR can all be jointly optimized for the speech recognition task. We perform E2E speech recognition experiments on the REVERB challenge dataset as well as on the VOiCES dataset. In these experiments, the proposed joint modeling approach yields significant improvements compared to the baseline E2E ASR system (average relative improvements of 21% on the REVERB challenge dataset and about 10% on the VOiCES dataset).

show abstract

Section: Reverb Challenge Asrsupporting

confidence: 88%

Section: Literature Reviewmentioning

confidence: 93%

Section: Signal Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

End-To-End Speech Recognition with Joint Dereverberation of Sub-Band Autoregressive Envelopes

Kumar

Purushothaman

Sreeram

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use FDLP features (Purushothaman et al, 2020) for far-field speech. This paper extends the prior work done in (Purushothaman et al, 2020b) by proposing a joint neural dereverberation which forms an elegant neural learning framework. Further, several ASR experiments with the joint modeling approach are also conducted in this work.…”

Section: Related Prior Workmentioning

confidence: 76%

Dereverberation of autoregressive envelopes for far-field speech recognition

Purushothaman

Sreeram

Kumar

et al. 2022

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

The task of speech recognition in far-field environments is adversely affected by the reverberant artifacts that elicit as the temporal smearing of the sub-band envelopes. In this paper, we develop a neural model for speech dereverberation using the long-term sub-band envelopes of speech. The sub-band envelopes are derived using frequency domain linear prediction (FDLP) which performs an autoregressive estimation of the Hilbert envelopes. The neural dereverberation model estimates the envelope gain which when applied to reverberant signals suppresses the late reflection components in the far-field signal. The dereverberated envelopes are used for feature extraction in speech recognition. Further, the sequence of steps involved in envelope dereverberation, feature extraction and acoustic modeling for ASR can be implemented as a single neural processing pipeline which allows the joint learning of the dereverberation network and the acoustic model. Several experiments are performed on the REVERB challenge dataset, CHiME-3 dataset and VOiCES dataset. In these experiments, the joint learning of envelope dereverberation and acoustic model yields significant performance improvements over the baseline ASR system based on log-mel spectrogram as well as other past approaches for dereverberation (average relative improvements of 10-24% over the baseline system). A detailed analysis on the choice of hyper-parameters and the cost function involved in envelope dereverberation is also provided.

show abstract

Utterance Weighted Multi-Dilation Temporal Convolutional Networks for Monaural Speech Dereverberation

Ravenscroft

Goetze

Hain

2022

2022 International Workshop on Acoustic Signal Enhancement (IWAENC)

View full text Add to dashboard Cite

show abstract

Deep Learning Based Dereverberation of Temporal Envelopes for Robust Speech Recognition

Cited by 6 publications

References 24 publications

End-To-End Speech Recognition with Joint Dereverberation of Sub-Band Autoregressive Envelopes

End-To-End Speech Recognition with Joint Dereverberation of Sub-Band Autoregressive Envelopes

Dereverberation of autoregressive envelopes for far-field speech recognition

Utterance Weighted Multi-Dilation Temporal Convolutional Networks for Monaural Speech Dereverberation

Contact Info

Product

Resources

About