Optimal splitting of HMM Gaussian mixture components with MMIE training

Normandin, Yves

doi:10.1109/icassp.1995.479625

Cited by 27 publications

(19 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Suppose that in (3) is defined to be the true probability of , so that only the terms and depend on the quality of the speech recognition model. Under this definition, the quantity is related by a constant to the model discriminant function [25], defined as (4) where (5) which is the likelihood ratio comparing the th word sequence hypothesis to the true word sequence . The discriminant function of a prosody dependent recognizer can be represented as (6) where (7) where is the prosody sequence that maximizes , and is the prosody hypothesis that maximizes .…”

Section: A Information-theoretic Analysismentioning

confidence: 99%

Prosody dependent speech recognition on radio news corpus of American English

Chen

Hasegawa‐Johnson

Cohen

et al. 2006

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-Does prosody help word recognition? This paper proposes a novel probabilistic framework in which word and phoneme are dependent on prosody in a way that reduces word error rates (WER) relative to a prosody-independent recognizer with comparable parameter count. In the proposed prosody-dependent speech recognizer, word and phoneme models are conditioned on two important prosodic variables: the intonational phrase boundary and the pitch accent. An information-theoretic analysis is provided to show that prosody dependent acoustic and language modeling can increase the mutual information between the true word hypothesis and the acoustic observation by exciting the interaction between prosody dependent acoustic model and prosody dependent language model. Empirically, results indicate that the influence of these prosodic variables on allophonic models are mainly restricted to a small subset of distributions: the duration PDFs (modeled using an explicit duration hidden Markov model or EDHMM) and the acoustic-prosodic observation PDFs (normalized pitch frequency). Influence of prosody on cepstral features is limited to a subset of phonemes: for example, vowels may be influenced by both accent and phrase position, but phrase-initial and phrase-final consonants are independent of accent. Leveraging these results, effective prosody dependent allophonic models are built with minimal increase in parameter count. These prosody dependent speech recognizers are able to reduce word error rates by up to 11% relative to prosody independent recognizers with comparable parameter count, in experiments based on the prosodically-transcribed Boston Radio News corpus.Index Terms-Acoustic model, ANN, duration, HMM, mutual information, pitch, prosody, ToBI, word error rate.

show abstract

Section: A Information-theoretic Analysismentioning

confidence: 99%

Prosody dependent speech recognition on radio news corpus of American English

Chen

Hasegawa‐Johnson

Cohen

et al. 2006

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…For the optimization of the objective function in Equation 4 we use the general purpose RPROP algorithm [12]. RPROP is a first order optimization algorithm that takes only the sign of the partial derivatives into account.…”

Section: B Optimization Proceduresmentioning

confidence: 99%

“…The Gaussian parameter splitting may also be accomplished discriminatively to obtain better fitting models, as in [4] where results on a digit recognition task are presented. The emphasis there is to retain the performance of a good system while successively reducing the number of parameters.…”

Section: Introductionmentioning

confidence: 99%

Discriminative splitting of Gaussian/log-linear mixture HMMs for speech recognition

Tahir

Schlüter

Ney

2011

2011 IEEE Workshop on Automatic Speech Recognition &Amp; Understanding

View full text Add to dashboard Cite

Abstract-This paper presents a method to incorporate mixture density splitting into the acoustic model discriminative log-linear training. The standard method is to obtain a high resolution model by maximum likelihood training and density splitting, and then further training this model discriminatively. For a single Gaussian density per state the log-linear MMI optimization is a global maximum problem, and by further splitting and discriminative training of this model we can get a higher complexity model. The mixture training is not a global maximum problem, nevertheless experimentally we achieve large gains in the objective function and corresponding moderate gains in the word error rate on a large vocabulary corpus

show abstract

“…But for discriminative training, these methods can not be directly applied and MLE is most commonly used for model initialization. Normandin [4] proposed an optimal splitting algorithm for GMDs with MMI criterion. But this algorithm was developed in a model complexity perspective and local optima were not of concern.…”

Section: Introductionmentioning

confidence: 99%

Gradient Boosting Learning of Hidden Markov Models

Zhao

2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings

View full text Add to dashboard Cite

In this paper, we present a new training algorithm, gradient boosting learning, for Gaussian mixture density (GMD) based acoustic models. This algorithm is based on a function approximation scheme from the perspective of optimization in function space rather than parameter space, i.e., stage-wise additive expansions of GMDs are used to search for optimal models instead of gradient descent optimization of model parameters. In the proposed approach, GMD starts from a single Gaussian and is built up by sequentially adding new components. Each new component is globally selected to produce optimal gain in the objective function. MLE and MMI are unified under the H-criterion, which is optimized by the extended BW (EBW) algorithm. A partial extended EM algorithm is developed for stage-wise optimization of new components. Experimental results on WSJ task demonstrate that the new algorithm leads to improved model quality and recognition performance.

show abstract

Optimal splitting of HMM Gaussian mixture components with MMIE training

Cited by 27 publications

References 5 publications

Prosody dependent speech recognition on radio news corpus of American English

Prosody dependent speech recognition on radio news corpus of American English

Discriminative splitting of Gaussian/log-linear mixture HMMs for speech recognition

Gradient Boosting Learning of Hidden Markov Models

Contact Info

Product

Resources

About