Bayesian update of dialogue state for robust dialogue systems

Abstract

This paper presents a new framework for accumulating beliefs in spoken dialogue systems. The technique is based on updating a Bayesian Network that represents the underlying state of a Partially Observable Markov Decision Process (POMDP). POMDP models provide a principled approach to handling uncertainty in dialogue but generally scale poorly with the size of the state and action space. The framework proposed, on the other hand, scales well and can be extended to handle complex dialogues. Learning is achieved with a factored summarising function that is applicable for many slot-filling type dialogues. The framework also provides a good structure from which to build hand-crafted policies. For very complex dialogues, this allows the POMDP's principled approach to uncertainty to be incorporated without requiring computationally intensive learning algorithms. Simulations show that the proposed framework outperforms standard techniques whenever errors increase.

Evaluation of a hierarchical reinforcement learning spoken dialogue system

“…This highlights the importance of effective and efficient mechanisms for dialogue history tracking. Future research can incorporate beliefs into the knowledge rich-states of the proposed framework with ideas from approaches such as regression methods , POMDPs (Williams, 2006), or Bayesian updates (Thomson et al, 2008).…”

Optimisation for POMDP-Based Spoken Dialogue Systems

“…All q f (τ j ) approximations are constrained to the Dirichlet distribution, with the parameters denoted by α f,j . The approximations for the other factors are fixed and the cavity distributions for the variables are defined as per (8). In the case of the discrete variables g t and g t−1 , the cavity distributions are computed by multiplying all factor approximations except forf .…”

“…Details can be found in [19]. The full algorithm operates by repeatedly choosing a factor to update, computing the cavity distributions in terms of the current approximations (8) and (9) and then updating the current approximating functions as per (10), (11) and (19). Similar to belief propagation, the process is repeated until changes in the approximating functions fall below a threshold.…”

“…Section 4 explains how a more general form of inference called Expectation Propagation(EP), which can be viewed as a form of expectation-maximisation, can be used for both belief tracking and parameter optimisation [5,6]. Section 5 explains how natural actor-critic reinforcement learning can be used to optimise the policy parameters P [7,8], and how with a simple extension, it can also be used to optimise the dialogue model parameters M [9]. Finally, section 6 addresses the problem of fast on-line policy optimisation using Gaussian processes as a non-parametric policy model [10,11,12].…”

Training and evaluation of the HIS POMDP dialogue system in noise

“…In the Bayesian Update of Dialogue State (BUDS) system, the user's goal is further factored into conditionally independent slots. The resulting system is then modelled as a dynamic Bayesian network (Thomson et al, 2008). A similar approach is also developed in (Bui et al, 2007a;Bui et al, 2007b).…”