Frustratingly Easy Noise-aware Training of Acoustic Models

Abstract

Environmental noises and reverberation have a detrimental effect on the performance of automatic speech recognition (ASR) systems. Multi-condition training of neural networkbased acoustic models is used to deal with this problem, but it requires many-folds data augmentation, resulting in increased training time. In this paper, we propose utterance-level noise vectors for noise-aware training of acoustic models in hybrid ASR. Our noise vectors are obtained by combining the means of speech frames and silence frames in the utterance, where the speech/silence labels may be obtained from a GMM-HMM model trained for ASR alignments, such that no extra computation is required beyond averaging of feature vectors. We show through experiments on AMI and Aurora-4 that this simple adaptation technique can result in 6-7% relative WER improvement. We implement several embedding-based adaptation baselines proposed in literature, and show that our method outperforms them on both the datasets. Finally, we extend our method to the online ASR setting by using frame-level maximum likelihood for the mean estimation.

Scenario Aware Speech Recognition: Advancements for Apollo Fearless Steps & CHiME-4 Corpora

“…In [20], the invariant representation learning technique was proposed, which demonstrated significant reduction in character error rate and robustness for out-ofdomain noise settings. In [21], a simple method was considered to extract a noise vector for acoustic model training. It is suggested that the technique could also be applied in online ASR by estimating the mean vector with frame-level maximum likelihood.…”

“…Factor aware training has been shown to be effective in ASR system development [5,19,21]. This training strategy produces a system that is more robust to factors such as noise, speaker, and room characteristics.…”