Use of syllable nuclei locations to improve ASR

Abstract

This work presents the use of dynamic Bayesian networks (DBNs) to jointly estimate word position and word identity in an automatic speech recognition system. In particular, we have augmented a standard Hidden Markov Model (HMM) with counts and locations of syllable nuclei. Three experiments are presented here. The first uses oracle syllable counts, the second uses oracle syllable nuclei locations, and the third uses estimated (non-oracle) syllable nuclei locations. All results are presented on the 10 and 500 word tasks of the SVitchboard corpus. The oracle experiments give relative improvements ranging from 7.0% to 37.2%. When using estimated syllable nuclei a relative improvement of 3.1% is obtained on the 10 word task.

Graphical models for integrating syllabic information

“…The improved baseline result of 58.6% reported in [Bartels and Bilmes, 2007] and Table 3 is due to a larger beam size. There were two differences that gave the improved result of 52.3% reported in [Bartels and Bilmes, 2008].…”

“…Switching parents can be used to specify context specific independence which can speed inference in some cases [Geiger and Heckerman, 1996] and can also be a convenience for the graph designer. Figure appeared in [Bartels and Bilmes, 2007] 3 Graphs Using Syllable Nuclei…”

“…After normalization, the PLP features are re-scaled to give the features a global mean and variance that is similar to the unnormalized features. The features used in [Livescu et al, 2007, Bartels andBilmes, 2007] had a similar scaling, but the values were numerically smaller and were not actually reflective of the unnormalized features. Second, the minimum occupancy count for the creation of a triphone cluster was increased and triphones that are never seen in training were completely removed from the clustering process.…”

“…This was implemented in an earlier version of the Word-Level graph and results were reported in Bartels and Bilmes [2007]. The soft integration was removed because using all of the detections was more effective, but certainly not all possibilities were exhausted and this concept could be revisited.…”

“…The acoustic observation variables are independent of past and future observation variables given their corresponding state, so acoustic cues can only affect duration and segmentation via the scoring of individual sub-phone states. (This example was generated by the method described in [Wang and Narayanan, 2005] for use in [Bartels and Bilmes, 2007], and maxima in the silence and unvoiced regions are removed. The neural-network based method used in the results here gives similar examples.…”