Integration of speech recognition and natural language processing in the MIT VOYAGER system

Zue, Victor W.; Glass, James; Goodine, David; Leung, Hoi Yan; Phillips, Michael; Polifroni, Joseph; Seneff, Stephanie

doi:10.1109/icassp.1991.150439

Cited by 26 publications

(16 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How these constraints are incorporated varies from estimating -gram probabilities from grammar-generated data [70] to computing a linear interpolation of the two models [43]. Most recently, syntactic information has been used specifically to determine equivalence classes on the -gram history, resulting in so-called dependency language models [19], [56], sometimes also referred to as structured language models [20], [42], [66].…”

Section: B Syntactically Driven Span Extensionmentioning

confidence: 99%

Exploiting latent semantic information in statistical language modeling

2000

View full text Add to dashboard Cite

Section: B Syntactically Driven Span Extensionmentioning

confidence: 99%

Exploiting latent semantic information in statistical language modeling

2000

View full text Add to dashboard Cite

“…The first attempts are based on using context free grammars (CFGs) [27,137,71]. The main contribution of structured LMs is started with Chelba et al [24] in which a dependency grammar framework with maximum entropy models is used to constrain the word prediction by the linguistically related words in the past.…”

Section: Structured Lmsmentioning

confidence: 99%

“…In [137], LMs are trained by generating syntactically plausible sentences by using a natural language component. Jurafsky et al [71] use stochastic CFGs (SCFGs) to extend the corpus for training and interpolates SCFG probabilities with bi-gram probabilities.…”

Section: Language Modelmentioning

confidence: 99%

Semantic language models with deep neural networks

Bayer

Riccardi

2016

Computer Speech & Language

View full text Add to dashboard Cite

Spoken language systems (SLS) communicate with users in natural language through speech. There are two main problems related to processing the spoken input in SLS. The first one is automatic speech recognition (ASR) which recognizes what the user says. The second one is spoken language understanding (SLU) which understands what the user means. We focus on the language model (LM) component of SLS. LMs constrain the search space that is used in the search for the best hypothesis. Therefore, they play a crucial role in the performance of SLS.It has long been discussed that an improvement in the recognition performance does not necessarily yield a better understanding performance. Therefore, optimization of LMs for the understanding performance is crucial. In addition, long-range dependencies in languages are hard to handle with statistical language models. These two problems are addressed in this thesis.We investigate two different LM structures. The first LM that we investigate enable SLS to understand better what they recognize by searching the ASR hypotheses for the best understanding performance. We refer to these models as joint LMs. They use lexical and semantic units jointly in the LM. The second LM structure uses the semantic context of an utterance, which can also be described as "what the system understands", to search for a better hypothesis that improves the recognition and the understanding performance. We refer to these models as semantic LMs (SELMs). SELMs use features that are based on a well established theory of lexical semantics, namely the theory of frame semantics. They incorporate the semantic features which are extracted from the ASR hypothesis into the LM and handle long-range dependencies by using the semantic relationships between words and semantic context. ASR noise is propagated to the semantic features, to suppress this noise we introduce the use of deep semantic encodings for semantic feature extraction. In this way, SELMs optimize both the recognition and the understanding performance.

show abstract

“…In addition, the use of combined knowledge sources can change computational efficiency by applying language constraints to predict possible next words, thus achieving significant pruning of the recognition search space. In this paper, we report primarily on experiments that change the shape of the search space; this work has been done on our loosely-coupled system using the N-best interface [11]. However, we also report briefly on the status of our experiments on the tight coupling of speech recognition and language understanding.…”

Section: Introductionmentioning

confidence: 99%

Integrating syntax and semantics into spoken language understanding

Hirschman¹,

Seneff²,

Goodine³

et al. 1991

Proceedings of the Workshop on Speech and Natural Language - HLT '91

View full text Add to dashboard Cite

This paper describes several experiments combining natural language and acoustic constraints to improve overall performance of the MIT VOYAGER spoken language system. This system couples the SUMMIT speech recognition system with the TINA language understanding system to answer spoken queries about navigational assistance in the Cambridge, MA, area. The overall goal of our research is to combine acoustic, syntactic and semantic knowledge sources. Our first experiment showed improvement by combining acoustic score and parse probability normalized for number of terminals. Results were further improved by the use of an explicit rejection criterion based on normalized parse probabilities. The use of the combined parse/acoustic score, together with the rejection criterion, gave an improvement in overall score of more than 33% on both training and test data, where score is defined as percent correct minus percent incorrect. Experiments on a fully integrated system which uses the parser to predict possible next words to the recognizer are now underway.

show abstract

Integration of speech recognition and natural language processing in the MIT VOYAGER system

Cited by 26 publications

References 8 publications

Exploiting latent semantic information in statistical language modeling

Exploiting latent semantic information in statistical language modeling

Semantic language models with deep neural networks

Integrating syntax and semantics into spoken language understanding

Contact Info

Product

Resources

About