Adaptation of Hybrid ANN/HMM Models Using Linear Hidden Transformations and Conservative Training

Abstract

A technique is proposed for the adaptation of automatic speech recognition systems using Hybrid models combining Artificial Neural Networks with Hidden Markov Models. The application of linear transformations not only to the input features, but also to the outputs of the internal layers is investigated. The motivation is that the outputs of an internal layer represent a projection of the input pattern into a space where it should be easier to learn the classification or transformation expected at the output of the network. A new solution, called Conservative Training, is proposed that compensates for the lack of adaptation samples in certain classes. Supervised adaptation experiments with different corpora and for different adaptation types are described. The results show that the proposed approach always outperforms the use of transformations in the feature space and yields even better results when combined with linear input transformations.

Cumulative Adaptation for BLSTM Acoustic Models

“…However, here only speaker independent input features were used. Other works in this field include [21,22,23,12,24] and [13], where feedforward neural networks were employed. I-vectors have been used sucessfully as a sole adaptation method using time-delay neural network (TDNN) [25] as well as BLSTM acoustic models for automatic speech recognition [26,27].…”

“…To avoid over-fitting, regularization such as in [10] is applied. Another approach is to insert and adapt speaker dependent linear layers into the network to transform either input feature [11], top-hidden-layer output [12], or hidden layer activations [13]. Finally, the acoustic model can be trained for different conditions separately such as in [14,15,16].…”

Fast speaker adaptation using extended diagonal linear transformation for deep neural networks

“…Previously, linear‐transformation methods have introduced an additional linear layer to a particular layer, such as the input and output layers, or to the top hidden layer. In and , an affine transformation has also been proposed to adapt the hidden layer by converting an additional scale matrix and bias vector.…”

A Bayesian view on acoustic model-based techniques for robust speech recognition

“…There are various approaches for adapting a DNN-HMM to changing acoustic environments or speaker characteristics. Most of them aim at either adapting certain weights of the DNN itself, such as in [74][75][76], or at presenting transformed/enriched input features to the DNN, as in [72,77]. In the following, we will briefly discuss the application of the Bayesian perspective, taken on in this article, to DNN-HMMs.…”