Agenda-based user simulation for bootstrapping a POMDP dialogue system

Abstract

This paper investigates the problem of bootstrapping a statistical dialogue manager without access to training data and proposes a new probabilistic agenda-based method for simulating user behaviour. In experiments with a statistical POMDP dialogue system, the simulator was realistic enough to successfully test the prototype system and train a dialogue policy. An extensive study with human subjects showed that the learned policy was highly competitive, with task completion rates above 90%.

Analysis of a new simulation approach to dialog system evaluation

“…The simulation approach followed in MeMo is similar to the agenda-based approach proposed by Schatzmann et al (2007a), in that utterances are generated from the description of the task from the user's perspective. Like in the previous approaches to dialog modeling described above, it assumes that the interaction (with any user interface) can be described as a sequence of states in which the system presents an interface and the user uses this interface to perform an action to proceed towards the task goal.…”

“…While Pietquin's model requires many parameters to be set, Schatzmann (2007a) introduces a simpler model, in which the behavior is elicited according to a stack-like agenda. In the beginning, the agenda is populated with actions derived from the user's goal.…”

Modeling the user state for context-aware spoken interaction in ambient assisted living

“…In [134], a new technique is presented for user modeling based on explicit representations of the user goal and the user agenda. The user agenda is a structure that contains the pending user dialog acts that are needed to elicit the information specifie in the goal.…”

A statistical approach to spoken dialog systems design and evaluation

“…In (Schatzmann et al, 2007a), a new technique for user simulation based on explicit representations of the user goal and the user agenda is presented. The user agenda is a structure that contains the pending user dialog acts that are needed to elicit the information specified in the goal.…”

“…Three dimensions are defined: high-level features (dialog and turn lengths), dialog style (speech-act frequency; proportion of goal-directed actions, grounding, formalities, and unrecognized actions; proportion of information provided, reprovided, requested and rerequested), and dialog efficiency (goal completion rates and times). The simulation presented in (Schatzmann et al, 2007a) is evaluated by testing the similarity between real and simulated data by means of statistical measures (dialog length, task completion rate and dialog perfor-6 mance).…”