Towards utterance-based neural network adaptation in acoustic modeling

Abstract

Despite the superior classification ability of deep neural networks (DNN), the performance of DNN suffers when there is a mismatch between training and testing conditions. Many speaker adaptation techniques have been proposed for DNN acoustic modeling but in case of environmental robustness the progress is still limited. It is also possible to use techniques developed for adapting speakers to handle the impact of environments at the same time, or to combine both approaches. Directly adapting the large number of DNN parameters is challenging when the adaptation set is small. The learning hidden unit contributions (LHUC) technique for unsupervised speaker adaptation of DNN introduces speaker dependent parameters to the existing speaker independent network to increase the automatic speech recognition (ASR) performance of the target speaker using small amounts of adaptation data. This paper investigates the LHUC to adapt the speech recognizer to target speakers and environments where the impacts of speakers and noise differences are quantified separately. Our finding shows that the LHUC is capable of adapting to both speaker and noise conditions at the same time. Compared to the speaker independent model, about 9% to 13% relative word error rate (WER) improvement are observed for all test conditions using AMI meeting corpus.

The CAPIO 2017 Conversational Speech Recognition System

“…Neural networks are well known to be hard for adaptation due to a huge number of parameters to be tuned, unlike statistical frameworks such as GMMs. As a result there have been alternative approaches to update only a small part of a neural network model [7,8,9,10] to obtain adaptation benefits. In this paper, we propose a simple model adaptation scheme exploiting parameter averaging.…”

“…To overcome this mismatch, neural network acoustic models need to be adapted, but it is widely known that they are not easy to be adapted due to a large number of parameters to be tuned. Most of the research effort on neural network adaptation thus has been focused either to update a part of parameters while fixing the rest [7,8,9,10] or to append domain-specific features (e.g., i-vectors [11] in case of speaker adaptation) for a better feature transformation [12,13,14]. In this paper, we propose a simple but generalized adaptation method for deep neural networks such that it can obtain expected adaptation benefits as well as avoids overfitting.…”

Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation

“…The results are reported in Table VIII: LHUC adaptation improves the accuracy in both experiments, although the gain for the SDM condition is smaller; how- 10 ever, the SDM system is characterised by twice as large WERs. Notice that LHUC has also been successfully applied to channel normalisation between distant and close talking microphones [75].…”

“…The results are reported in ever, the SDM system is characterised by twice as large WERs. Notice that LHUC has also been successfully applied to channel normalisation between distant and close talking microphones [75].…”

Densely Connected Networks for Conversational Speech Recognition

“…Neural networks are well known to be hard for adaptation due to a huge number of parameters to be tuned, unlike statistical frameworks such as GMMs. As a result there have been alternative approaches to update only a small part of a neural network model [20,21,22,23] to obtain adaptation benefits. In this paper, we utilizes a simple model adaptation scheme exploiting parameter averaging.…”