Pronunciation modeling using a finite-state transducer representation

Abstract

The MIT summit speech recognition system models pronunciation using a phonemic baseform dictionary along with rewrite rules for phonological variation and multi-word reductions. Each pronunciation component is encoded within a finite-state transducer (FST) representation whose transition weights can be trained using an EM algorithm for finite-state networks. This paper details our modeling approach and demonstrates its benefits and weaknesses, both conceptually and empirically, using the recognizer for our jupiter weather information system. Experiments show that the use of phonological rules within our system achieves word error rate reductions between 4% and 9% over different test sets when compared against a system using no phonological rules. The same FST representation can also be used in generative mode within a concatenative speech synthesizer.

Subword Modeling for Automatic Speech Recognition: Past, Present, and Emerging Approaches

“…In particular, the interactions between subword modeling, observation modeling, and the choice of acoustic observations deserve more study. For example, phonetic dictionary expansion may affect different systems differently (e.g., possibly achieving greater improvements in a segment-based recognizer [33] than in HMM-based recognizers [30], [10]), but to our knowledge there have been no direct comparisons on identical tasks and data sets. We have also only briefly touched on automatic sub-word unit learning and the related task of automatic dictionary learning [39], [40], [47].…”

“…This led to a great deal of activity on modeling pronunciation variation, including two workshops sponsored by the International Speech Communication Association [27], [28]. The majority (but by no means all) of the proposed approaches during this period kept the phone as the basic sub-word unit, and focused on ways of predicting the possible phonetic sequences for any given word using phonological rules or other means [29], [30], [31], [32], [33], [7], [34], [10].…”

“…For example, the flapping of [t] in little could be generated by the rule { ih t ax → dx } (read "[t] can be realized as a flap between the vowels [ih] and [ax]"). Such rules can be specified manually from linguistic knowledge [33] or learned automatically, typically using decision trees [30], [10]. Once a set of rules is specified or learned, it can be used to expand a dictionary to form a single new dictionary (a static expansion) or to expand the dictionary dynamically during recognition in response to unpredictable context such as the speaking rate [10].…”

“…Phonetic dictionary expansion has produced improvements in some systems [30], [10], [33]. However, the improvements have been more modest than hoped, considering the very large difference in performance on read and conversational renditions of the same word sequences [5].…”

“…is typically not learned jointly with the other parameters, but rather decision trees are separately learned for predicting the phone distribution given the context variables [30], [10]. In other work, rule "firing" probabilities are learned separately or as part of the complete recognizer [33]. In addition, the same model structure can describe certain acousticsbased models; for example, Hain's HMS-HMM approach (Section II-C) [48] has the same structure except that the "surface phone state" is an abstract HMM model state, and is not shared across canonical phones with similar surface realizations.…”