Combining TDNN and HMM in a hybrid system for improved continuous-speech recognition

Abstract

The paper presents a hybrid continuous-speech recognition system that leads to improved results on the speaker dependent DARPA Resource Management task. This hybrid system, called the combined system, is based on a combination of normalized neural network output scores with hidden Markov model (HMM) emission probabilities. The neural network is trained under mean square error and the HMM is trained under maximum likelihood estimation. In theory, whatever criterion may be used, the same word error rate should be reached if enough training data is available. As this is never the case, the idea of combining two different criteria, each of them extracting complementary characteristics of the feature is interesting. A state-of-the-art HMM system will be combined with a time delay neural network (TDNN) integrated in a Viterbi framework. A hierarchical TDNN structure is described that splits training into subtasks corresponding to subsets of phonemes. This structure makes training of TDNNs on large-vocabulary tasks manageable on workstations. It will be shown that the combined system, despite the low accuracy of the hierarchical TDNN, achieves a word error rate reduction of 15% with respect to our state-of-the-art HMM system. This reduction is obtained with a 10% increase only in the number of parameters.

Revisiting hybrid and GMM-HMM system combination techniques

“…These investigations, mainly based on model averaging, showed some success when combing contextindependent hybrid systems based on multi-layer perceptrons (MLPs) and recurrent neural networks (RNNs). Dugast et al [16] combined posterior probability estimates obtained from a time-delay neural network with the likelihoods generated by an HMM system with state emissions modelled by a mixture of Laplacians. Similar approaches combining scaledlikelihoods produced by a two-layer MLP and HMM-GMM likelihoods were also investigated [17].…”

A survey of hybrid ANN/HMM models for automatic speech recognition

“…The proposed technique allowed for a signi"cant gain in performance for some classes of confusable letters (e.g.,`Ba,`Da, and`Va), from 491 spellings of names collected over the telephone channel. The approach proposed by [31] is not truly a hybrid, since it uses a standard HMM (trained with Viterbi algorithm on the ML criterion), which is used in parallel with connectionist models by combining the estimates of the emission probabilities (likelihoods) provided by the HMM with the normalized scores obtained with the ANNs. The linear combination scheme is the following:…”

“…Parallel ANN/HMM state-probability estiamtes [31] HMM estimates of the emission probabilities are linearly combined with scores obtained with a hierarchical mixture of TDNNs.…”

Speech Processing, Recognition and Artificial Neural Networks

“…A part of the prosodic information is 88 obviously linguistic, but the rest of it conveys non-linguistic information. In our flowchart this branch is fuzzier than the other: levels and units are less clearly defined.…”