Currently, first-order hidden Markov models (HMMs) form the backbone around which most automatic speech processing applications are built. Their higher-order extensions are known to be more powerful, but, due to their complexity and computational demands, they are seldomly used. It is the purpose of this work to advance their application In this work we unify HMMs of all orders by deriving and proving the ORder rEDucing (ORED) algorithm. This algorithm will reduce any higher-order HMM (also mixed-order) to an equivalent first-order representation. This makes it possible to process any higher-order HMM using known first-order algorithms, thereby making unnecessary the current approach of extending specific HMM algorithms to specific higher orders. It also provides an alternative theoretical basis to reason about high-order HMMs. From this perspective high-order transition probabilities are simply powerful mathematical specifications of first-order topology. We use this insight to explain old topologies and to design new ones.We address computational concerns by developing the Fast Incremental Training (FIT) algorithm. This algorithm avoids training redundant high-order probabilities by noting which lower-order transition probabilities are zero. This considerably reduces the memory and processor requirements during training. In addition, the resultant models have far fewer parameters and generalise better on previously unseen data.To show the practical applicability of our methodology we apply it to automatic language recognition. We find that it compares well with systems that require expensive transcribed databases (our system does not require this).
ContentsCHAPTER 1 PURPOSE, OBJECTIVES AND CONTRIBUTION .