Some improvements in speech recognition algorithms based on HMM

Efficient training of high-order hidden Markov models using first-order representations

“…As noted in Section 1, Kriouile et al (1990) developed enhanced training algorithms for second-order HMMs. Using ORED, the HMM can also be trained by using standard first-order optimization algorithms.…”

“…Note that, unlike the HMM1, both the previous and the current state determine the correct choice amongst the number of alternative probabilities on a given transition. Typically these probabilities are represented in a cubic structure, and the calculation of f(X L 1 |M) requires specialized algorithms to track the longer history of states (Kriouile et al, 1990).…”

“…In previous work, both the Viterbi (He, 1988) and the Baum-Welch (Kriouile, Mari & Haton, 1990) algorithms have been extended to second-order models. The enhanced duration modelling of second-order HMMs, compared to first-order models, improves the accuracy obtained in digit and letter recognition experiments (Mari & Haton, 1994;Mari, Fohr & Junqua, 1996;Mari, Haton & Kriouile, 1997).…”

Comparison of the order reducing (ORED) and fast incremental training (FIT) algorithms for training high order hidden Markov models

“…For training higher order HMMs, only second order extensions are currently available [4,5,6], where-as the ORED approach allows all existing algorithms used for processing first order HMMs to be directly applied to higher order HMMs. We therefor expects identical results using either, training algorithms extended to higher order HMMs, or the ORED approach.…”

“…One approach to this, would be to enlarge the 2-dimensional transition matrix used for first order HMMs to be a p-dimensional transition matrix for pth order HMMs. The Baum-Welch and Viterbi algorithms can then be expanded to optimise these higher order models [4,5,6], In contrast, the ORED algorithm first reduces the higher order model to an equivalent first order model, and then trains it using an existing algorithm [1,2]. It does this by recognising that, if all pairs of states that are joined by transitions are by a new unique state, and transitions are correspondingly mapped between these new states, the result will be a first order HMM equivalent to the original second order HMM.…”