Effects of the user model on simulation-based learning of dialogue strategies

Schatztnann, J.; Stuttle, Matt; Weilhammer, Karl; Young, Steve

doi:10.1109/asru.2005.1566539

Cited by 54 publications

(44 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting learned policy was shown to have the best performance of all systems tested. It is a well-documented problem that performance with simulations and real users are not always comparable [8]. Future work will need to evaluate the performance of the system with real users.…”

Section: Discussionmentioning

confidence: 99%

Bayesian update of dialogue state for robust dialogue systems

Thomson

Schatzmann

Young

2008

2008 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

This paper presents a new framework for accumulating beliefs in spoken dialogue systems. The technique is based on updating a Bayesian Network that represents the underlying state of a Partially Observable Markov Decision Process (POMDP). POMDP models provide a principled approach to handling uncertainty in dialogue but generally scale poorly with the size of the state and action space. The framework proposed, on the other hand, scales well and can be extended to handle complex dialogues. Learning is achieved with a factored summarising function that is applicable for many slot-filling type dialogues. The framework also provides a good structure from which to build hand-crafted policies. For very complex dialogues, this allows the POMDP's principled approach to uncertainty to be incorporated without requiring computationally intensive learning algorithms. Simulations show that the proposed framework outperforms standard techniques whenever errors increase.

show abstract

Section: Discussionmentioning

confidence: 99%

Bayesian update of dialogue state for robust dialogue systems

Thomson

Schatzmann

Young

2008

2008 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

show abstract

“…More evidence of the adequacy of the automatic annotations is that the resulting data has been used to learn successful dialogue strategies (Henderson et al 2005(Henderson et al , 2008Frampton and Lemon 2006;Lemon et al 2006a), and high quality user simulations (Georgila et al , 2006Schatzmann et al 2005aSchatzmann et al , 2005b. The final annotated corpus will be made publicly available, for use by other researchers.…”

Section: Discussionmentioning

confidence: 99%

“…Both preliminary versions of our annotations and the version that we present here have been used successfully in Georgila et al (2005aGeorgila et al ( , 2006, Henderson et al (2005Henderson et al ( , 2008, Schatzmann et al (2005aSchatzmann et al ( , 2005b, Frampton and Lemon (2006). Note that prior work on dialogue context annotations (Poesio et al 1999) was not automated, and was not suitable for large-scale annotations.…”

Section: The 'Information State Update' Approachmentioning

confidence: 98%

“…With regard to perplexity again the user model trained on the automatic annotations performs better. Another indirect indication of the good quality of the annotations and proof of their usefulness is that they have been used to train successful dialogue strategies (Henderson et al 2005(Henderson et al , 2008Frampton and Lemon 2006) and user simulations in Georgila et al (2005aGeorgila et al ( , 2006 and Schatzmann et al (2005aSchatzmann et al ( , 2005b.…”

Section: Indirect Evaluation Via User Simulationsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic annotation of context and speech acts for dialogue corpora

Georgila¹,

Lemon²,

Henderson³

et al. 2009

Nat. Lang. Eng.

View full text Add to dashboard Cite

Richly annotated dialogue corpora are essential for new research directions in statistical learning approaches to dialogue management, context-sensitive interpretation, and contextsensitive speech recognition. In particular, large dialogue corpora annotated with contextual information and speech acts are urgently required. We explore how existing dialogue corpora (usually consisting of utterance transcriptions) can be automatically processed to yield new corpora where dialogue context and speech acts are accurately represented. We present a conceptual and computational framework for generating such corpora. As an example, we present and evaluate an automatic annotation system which builds 'Information State Update' (ISU) representations of dialogue context for the Communicator (2000 and 2001) corpora of human-machine dialogues (2,331 dialogues). The purposes of this annotation are to generate corpora for reinforcement learning of dialogue policies, for building user simulations, for evaluating different dialogue strategies against a baseline, and for training models for contextdependent interpretation and speech recognition. The automatic annotation system parses system and user utterances into speech acts and builds up sequences of dialogue context representations using an ISU dialogue manager. We present the architecture of the automatic annotation system and a detailed example to illustrate how the system components interact to produce the annotations. We also evaluate the annotations, with respect to the task completion metrics of the original corpus and in comparison to hand-annotated data and annotations produced by a baseline automatic system. The automatic annotations perform well and largely outperform the baseline automatic annotations in all measures. The resulting annotated corpus has been used to train high-quality user simulations and to learn successful dialogue strategies. The final corpus will be made publicly available.

show abstract

“…This paper adopts a cross-model approach to evaluation whereby policies are trained and tested on different user models, as previously suggested by [33]. To provide a competitive baseline for comparisons with the trained agenda model, a handcrafted simulator [34] was designed to reproduce user behavior for the given tourist information domain as naturally as possible.…”

Section: A Cross-model Evaluation With a Handcrafted Baseline Simulatormentioning

confidence: 99%

The Hidden Agenda User Simulation Model

Schatzmann

Young

2009

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-A key advantage of taking a statistical approach to spoken dialogue systems is the ability to formalise dialogue policy design as a stochastic optimization problem. However, since dialogue policies are learnt by interactively exploring alternative dialogue paths, conventional static dialogue corpora cannot be used directly for training and instead, a user simulator is commonly used. This paper describes a novel statistical user model based on a compact stack-like state representation called a user agenda which allows state transitions to be modeled as sequences of push-and pop-operations and elegantly encodes the dialogue history from a user's point of view. An expectation-maximisation based algorithm is presented which models the observable user output in terms of a sequence of hidden states and thereby allows the model to be trained on a corpus of minimally annotated data. Experimental results with a real-world dialogue system demonstrate that the trained user model can be successfully used to optimise a dialogue policy which outperforms a hand-crafted baseline in terms of task completion rates and user satisfaction scores.Index Terms-Dialogue management, Markov decision process, planning under uncertainty, spoken dialogue system (SDS), user simulation.

show abstract

Effects of the user model on simulation-based learning of dialogue strategies

Cited by 54 publications

References 5 publications

Bayesian update of dialogue state for robust dialogue systems

Bayesian update of dialogue state for robust dialogue systems

Automatic annotation of context and speech acts for dialogue corpora

The Hidden Agenda User Simulation Model

Contact Info

Product

Resources

About