Combining POMDPs trained with user simulations and rule-based dialogue management in a spoken dialogue system

Abstract

Over several years, we have developed an approach to spoken dialogue systems that includes rule-based and trainable dialogue managers, spoken language understanding and generation modules, and a comprehensive dialogue system architecture. We present a Reinforcement Learning-based dialogue system that goes beyond standard rule-based models and computes on-line decisions of the best dialogue moves. The key concept of this work is that we bridge the gap between manually written dialog models (e.g. rule-based) and adaptive computational models such as Partially Observable Markov Decision Processes (POMDP) based dialogue managers.

A Case Study of Applying Decision Theory in the Real World

“…Methods have been developed for incorporating business rules into the policy, encoding structured domain knowledge into the state, and learning from a high-fidelity simulated user. These systems have been demonstrated to the research community [92,31,80,73] and are available for public use [91,75]. Recently, toolkits have been released to assist non-experts build statistical dialog systems [12,83].…”