Robust spoken language understanding in a computer game

Boye, Johan; Gustafson, Joakim; Wirén, Mats

doi:10.1016/j.specom.2005.06.015

“…The DM is based on work described previously (Boye & Gustafson, 2005;Boye, Gustafson, & Wirén, 2006;Boye, 2007), but has been substantially modified for the challenges of conversational dialogue. It receives user utterances from the NLU as semantic representa-tions (right side of Figure 3).…”

Section: Dialogue Managementmentioning

confidence: 99%

Interaction Strategies for an Affective Conversational Agent

Smith

¹

,

Crook

²

,

Charlton

³

et al. 2011

Presence: Teleoperators and Virtual Environments

Self Cite

View full text Add to dashboard Cite

The development of embodied conversational agents (ECA) as companions brings several challenges for both affective and conversational dialogue. These include challenges in generating appropriate affective responses, selecting the overall shape of the dialogue, providing prompt system response times, and handling interruptions. We present an implementation of such a companion showing the development of individual modules that attempt to address these challenges. Further, to resolve resulting conflicts, we present encompassing interaction strategies that attempt to balance the competing requirements along with dialogues from our working prototype to illustrate these interaction strategies in operation. Finally, we provide the results of an evaluation of the companion using an evaluation methodology created for conversational dialogue and including analysis using appropriateness annotation.

show abstract

“…The DM is based on work described previously (Boye & Gustafson, 2005;Boye, Gustafson, & Wirén, 2006;Boye, 2007), but has been substantially modified for the challenges of conversational dialogue. It receives user utterances from the NLU as semantic representations (right side of Figure 3).…”

Section: Dialogue Managementmentioning

confidence: 99%

Interaction Strategies for an Affective Conversational Agent

Smith

¹

,

Crook

²

,

Boye

³

et al. 2010

Intelligent Virtual Agents

Self Cite

View full text Add to dashboard Cite

The development of embodied conversational agents (ECA) as companions brings several challenges for both affective and conversational dialogue. These include challenges in generating appropriate affective responses, selecting the overall shape of the dialogue, providing prompt system response times, and handling interruptions. We present an implementation of such a companion showing the development of individual modules that attempt to address these challenges. Further, to resolve resulting conflicts, we present encompassing interaction strategies that attempt to balance the competing requirements along with dialogues from our working prototype to illustrate these interaction strategies in operation. Finally, we provide the results of an evaluation of the companion using an evaluation methodology created for conversational dialogue and including analysis using appropriateness annotation.

show abstract

“…Gallwitz et al [24] also found in sentences' prosodic limits a new manner to segment and identify words. Lastly, prosody helps in dialog acts determination without independent syntactic and semantic analysis [25] and in games control contextualization [26] [27].…”

Section: Related Workmentioning

confidence: 99%

A Connectionist Approach to Speech Understanding

Müller

¹

,

Siqueira

²

,

Navaux

³

2006

The 2006 IEEE International Joint Conference on Neural Network Proceedings

View full text Add to dashboard Cite

This paper argues that connectionist systems are a good approach to implement a speech understanding computational model. In this direction, we propose SUM, a Speech Understanding Model, which is a software architecture based on recent neurocognitive researches. The SUM´s computational implementation applies wavelets transforms to speech signal processing and connectionist models to syntactic parsing and prosodic-semantic mapping. This approach enables a system of analysis by compensation, when the syntactic analysis does not offer a good reliable level, it is possible to evaluate prosodicsemantic analysis, such as in human speech understanding. I. INTRODUCTIONThe studies of computational systems, which research oral human-computer communication, are commonly called Spoken Language Understanding (SLU) Systems. This kind of system is derived from two joint technologies: Automatic Speech Recognition (ASR) and Natural Language Processing (NLP). These two technologies are complementary. The natural language can help the speech recognition through information on syntax and semantics. Furthermore, the speech recognition can improve the understanding of the language with contextual information, such as the intonation of the words (prosody) [1].Current SLU systems lack technology based on biological studies and there is large interference of the designer in the system configuration. Therefore, it is necessary a study of the biological processes involved in the speech understanding, which will lead to a coherent and valid computational representation of such processes. Our work supports the possibility of computational systems integration to represent the speech understanding process, which will improve the SLU systems development.In fact, recent neurocognitive researches present new approaches to base computational representations. The neurocognitive model developed by Friederici [2] demonstrates a parallel and complementary process that is performed by syntactic and semantics analyses. In this direction, we have proposed a new computational architecture to allow a more robust approach to spoken language processing [3] [4].This work argues that it is possible to unify several computational systems to represent the speech understanding process. Thus, we propose SUM (figure 1), a Speech Understanding Model based on a neurocognitive model of auditory sentence (section III). Through SUM, we search a computational model that works by compensation: when syntactic analysis fails, semantic analysis can be successful, and vice versa.

show abstract

Robust spoken language understanding in a computer game

Cited by 11 publications

References 12 publications

Interaction Strategies for an Affective Conversational Agent

Interaction Strategies for an Affective Conversational Agent

Interaction Strategies for an Affective Conversational Agent

A Connectionist Approach to Speech Understanding

Contact Info

Product

Resources

About