Backoff model training using partially observed data

Ji, Gang; Bilmes, Jeff

doi:10.3115/1220835.1220871

Cited by 17 publications

(18 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The convergence (5) then follows immediately from (14). The convergence (5) then follows immediately from (14).…”

Section: Known Weightsmentioning

confidence: 84%

“…, S K g as in (12) and (13) and the empirical measureŝ P l n (u j , p j ) as in (14). We now summarize the above by giving a formal definition of the adjusted VT with the weight correction.…”

Section: Unknown Weightsmentioning

confidence: 99%

“…Motivated by applications of the Viterbi training (VT) algorithm to estimate parameters of hidden Markov models (HMM) in speech recognition (Huang, Ariki, and Jack [11], Ney, Steinbiss, Haeb-Umbach, Tran, and Essen [26], Rabiner and Juang [31], Rabiner, Wilpon, and Juang [32], Steinbiss et al [36], Ström, Hetherington, Hazen, Sandness, and Glass [37]), natural language models (Ji and Bilmes [14], Och and Ney [27]), image analysis (Joshi, Li, and Wang [15], Li, Gray, and Olshen [22]), bioinformatics (Ehret et al [8], Ohler, Niemann, Liao, and Rubin [28]), and gene discovery via unsupervized learning (Lomsadze, Ter-Hovhannisyan, Chernoff, and Borodovsky [24]), we propose a new principled way to improve the accuracy of the VT estimators while preserving the essential computational advantages of the baseline algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adjusted Viterbi Training

Lember

Koloydenko

2007

Prob. Eng. Inf. Sci.

View full text Add to dashboard Cite

Viterbi training (VT) provides a fast but inconsistent estimator of hidden Markov models (HMM). The inconsistency is alleviated with a little extra computation when we enable VT to asymptotically fix the true values of the parameters. This relies on infinite Viterbi alignments and associated with them limiting probability distributions. First in a sequel, this article is a proof of concept; it focuses on mixture models, an important but special case of HMM where the limiting distributions can be calculated exactly. A simulated Gaussian mixture shows that our central algorithm (VA1) can significantly improve the accuracy of VT with little extra cost. Next in the sequel, we present elsewhere a theory of the adjusted VT for the general HMMs, where the limiting distributions are more challenging to find. Here, we also present another, more advanced correction to VT and verify its fast convergence and high accuracy; its computational feasibility requires additional investigation.

show abstract

“…The convergence (5) then follows immediately from (14). The convergence (5) then follows immediately from (14).…”

Section: Known Weightsmentioning

confidence: 84%

Section: Unknown Weightsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adjusted Viterbi Training

Lember

Koloydenko

2007

Prob. Eng. Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Over the last years, dynamic Bayesian networks have been investigated for many sequential data modeling tasks (automatic speech recognition, POS and dialog-act tagging [9], DNA sequence analy- sis...). DBN have shown a great flexibility for complex stochastic system representation and good performance are generally observed when compared to other standard techniques.…”

Section: Dbn-based Understanding Modelsmentioning

confidence: 99%

Dynamic Bayesian Networks and Discriminative Classifiers for Multi-Stage Semantic Interpretation

Lefvre

2007

2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07

View full text Add to dashboard Cite

In this paper, a multi-stage spoken language understanding system is presented. This stochastic module is for the first time based on a combination of dynamic Bayesian networks and conditional random field classifiers. The former generative models allow to derive basic concept sequences from the word sequences which are in turn augmented with modalities and hierarchical information by the latter discriminative models. To provide efficiently smoothed conditional probability estimates, factored language models with a generalized parallel backoff procedure are used as the network edge implementation. This framework allows a great flexibility in terms of probability representation facilitating the development of the stochastic levels (semantic and lexical) of the system.Experiments are carried out on the French MEDIA task (tourist information and hotel booking). The MEDIA 10k-utterance training corpus is conceptually rich (more than 80 basic concepts) and is provided with a manually segmented annotation. On this complex task, the proposed multi-stage system is shown to offer better performance than the MEDIA'05 evaluation campaign best system [1].

show abstract

“…Over the last years, dynamic Bayesian networks have been investigated for many sequential data modeling tasks (automatic speech recognition, POS and Dialog-Act tagging [10], DNA sequence analysis...). DBN have shown a great flexibility for complex stochastic system representation and good performance are generally observed when compared to other standard techniques.…”

Section: Dbn-based Slu Modelsmentioning

confidence: 99%

A DBN-Based Multi-Level Stochastic Spoken Language Understanding System

Lefèvre

2006

2006 IEEE Spoken Language Technology Workshop

View full text Add to dashboard Cite

In a previous paper [1], extensions of the 2-level stochastic speech understanding system have been proposed. Firstly the 3-level system is obtained through the introduction of a stochastic concept value normalization module. Then the 2+1-level system is obtained as a degraded 3-level system where the conceptual decoding and value normalization steps are decoupled, thus allowing to greatly reduce the model complexity and improve its trainability.In this paper, a multi-level spoken language understanding system is presented. This stochastic module is for the first time based on dynamic Bayesian networks. Factored language models with a generalized parallel backoff procedure are used as edge implementation to provide efficiently smoothed conditional probability estimates. This framework allows a great flexibility in terms of probability representation facilitating the development of the stochastic levels of the system.The proposed approaches, 3-level and 2+1-level, are evaluated on the French MEDIA task (tourist information and hotel booking). The MEDIA 10k-utterance training corpus is segmentally annotated, allowing a direct training of the various levels of the conceptual models. The best DBN-based system obtains performance comparable to those of the MEDIA'05 evaluation campaign best system [2].

show abstract

Backoff model training using partially observed data

Cited by 17 publications

References 24 publications

Adjusted Viterbi Training

Adjusted Viterbi Training

Dynamic Bayesian Networks and Discriminative Classifiers for Multi-Stage Semantic Interpretation

A DBN-Based Multi-Level Stochastic Spoken Language Understanding System

Contact Info

Product

Resources

About