Training and evaluation of the HIS POMDP dialogue system in noise

Abstract

This paper investigates the claim that a dialogue manager modelled as a Partially Observable Markov Decision Process (POMDP) can achieve improved robustness to noise compared to conventional state-based dialogue managers. Using the Hidden Information State (HIS) POMDP dialogue manager as an exemplar, and an MDP-based dialogue manager as a baseline, evaluation results are presented for both simulated and real dialogues in a Tourist Information Domain. The results on the simulated data show that the inherent ability to model uncertainty, allows the POMDP model to exploit alternative hypotheses from the speech understanding system. The results obtained from a user trial show that the HIS system with a trained policy performed significantly better than the MDP baseline.

Optimisation for POMDP-Based Spoken Dialogue Systems

“…Although (28) can be used to optimise the policy parameters, the use of the "plain" gradient yields rather poor convergence properties since methods using this gradient often suffer from extremely flat plateaus in the expected reward function. In contrast, the natural gradient defined as…”

“…An appealing feature of these algorithms is that in practice the Fisher Information Matrix does not need to be explicitly computed. Inspecting (28), it can be observed that the expression…”

“…However, in order to make real-world dialogue tasks tractable, the belief space must be compressed into a so-called summary space which leads to similar approximation issues. Furthermore, even when compressed to a few hundred grid points, around 10 5 dialogues are still required for optimisation [28]. This number is too large to allow training with real users so the interaction is normally performed with a simulated user.…”

Back-off action selection in summary space-based POMDP dialogue systems

“…Training of the fixed back-off system starts in low noise and then incrementally increases it, as in [12]. Approximately 1, 000, 000 dialogues were used for training and the total number of grid points was 400.…”

Continuous asr for flexible incremental dialogue

“…This gives rise to a rigid model of turntaking, which can be unnatural to users. There are many conditions under which users employ a more flexible turn-taking model, for example when they are under cognitive load and use more fillers, hesitations and barge-ins [1]. Furthermore, rigid turn-taking models often rely on a voice activity detection (VAD) component to decide whether the user is speaking or not, and this component can perform poorly, especially in noisy conditions, leading to confusion if the speech/non-speech classification is incorrect.…”