Subspace LHUC for Fast Adaptation of Deep Neural Network Acoustic Models

Abstract

Recently, the learning hidden unit contributions (LHUC) method is proposed for the adaptation of deep neural network (DNN) based acoustic models for automatic speech recognition (ASR). In LHUC, a set of speaker dependent (SD) parameters is estimated to linearly recombine the hidden units in an unsupervised fashion. Although LHUC performs considerably well, the gains diminish when the availability of the adaptation data amount decreases. Moreover, the per-speaker footprint of LHUC adaptation is in thousands and it is not desirable. Therefore, in this work, we propose the subspace LHUC, where the SD parameters are estimated in a subspace and connected to various layers through a new set of adaptively trained weights. We evaluate the subspace LHUC in the Aurora4 and AMI IHM tasks. Experimental results show that the subspace LHUC outperforms standard LHUC adaptation. With utterance-level fast adaptation, the subspace LHUC achieved 11.3% and 4.5% relative improvements over the standard LHUC for the Aurora4 and AMI IHM tasks respectively. Furthermore, the subspace LHUC reduces the per-speaker footprint by 94% over the standard LHUC adaptation.

Auxiliary Feature Based Adaptation of End-to-end ASR Systems

“…Future work will include investigations with larger English corpora, other approaches for exploiting the auxiliary features [31,32] as well as integration into an online decoding scheme [38].…”

“…Much research has been performed on auxiliary feature based adaptation, proposing different auxiliary features [18,19,30] and model architectures [18,31,32]. Auxiliary featurebased adaptation has also been proposed recently for CTCbased multilingual E2E ASR system using an auxiliary feature representing a target language [20].…”

Feature Based Domain Adaptation for Neural Network Language Models with Factorised Hidden Layers

“…A slightly different approach for model based adaptation uses a gating mechanism. In acoustic model adaptation, a concept called learning hidden unit contributions (LHUC) [31], [32] has previously been introduced. In this case, the speaker adaptation data is used to apply a gating mechanism on the hidden units in a neural network based acoustic model.…”

“…In [20] the adaptation was applied as a model based adaptation of the RNN. We will, however, use LHUC based adaptation as a feature based adaptation method, where the adaptation weights are calculated from auxiliary features in a similar way to [32]. We use LHUC in a similar scheme to fLHN-LSTM, as shown in Fig.…”

“…We will denote LHUC based model adaptation as fLHUC-LSTM. The LDA features are multiplied with a linear layer, followed by a sigmoid non-linearity and a weighting by two (as proposed by [31], [32])…”