A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimization

Daubigney, Lucie; Geist, Matthieu; Chandramohan, Senthilkumar; Pietquin, Olivier

doi:10.1109/jstsp.2012.2229257

Cited by 49 publications

(36 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Standard Qlearning (Watkins, 1989) has also been tested as an online algorithm but unsuccessfully, which is compliant with previous works (e.g. (Daubigney et al, 2012)). …”

Section: Learningsupporting

confidence: 66%

Optimising Turn-Taking Strategies With Reinforcement Learning

Khouzaimi

Laroche

Lefèvre

2015

Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

In this paper, reinforcement learning (RL) is used to learn an efficient turn-taking management model in a simulated slotfilling task with the objective of minimising the dialogue duration and maximising the completion task ratio. Turn-taking decisions are handled in a separate new module, the Scheduler. Unlike most dialogue systems, a dialogue turn is split into microturns and the Scheduler makes a decision for each one of them. A Fitted Value Iteration algorithm, Fitted-Q, with a linear state representation is used for learning the state to action policy. Comparison between a non-incremental and an incremental handcrafted strategies, taken as baselines, and an incremental RL-based strategy, shows the latter to be significantly more efficient, especially in noisy environments.

show abstract

“…Standard Qlearning (Watkins, 1989) has also been tested as an online algorithm but unsuccessfully, which is compliant with previous works (e.g. (Daubigney et al, 2012)). …”

Section: Learningsupporting

confidence: 66%

Optimising Turn-Taking Strategies With Reinforcement Learning

Khouzaimi

Laroche

Lefèvre

2015

Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

show abstract

“…The choice of policy learning algorithm is important because learning POMDP policies is challenging and dialog applications exhibit properties not often encountered in other reinforcement learning applications (Daubigney et al, 2012). We use KTD-Q (Kalman Temporal Difference Qlearning (Geist and Pietquin, 2010)) to learn the dialog policy as it was designed to satisfy some of these properties and tested in a dialog system with simulated users (Pietquin et al, 2011).…”

Section: Dialog Strategy Learningmentioning

confidence: 99%

Integrated Learning of Dialog Strategies and Semantic Parsing

Padmakumar

Thomason

Mooney

2017

Proceedings of the 15th Conference of the European Chapter of The Association for Computational Linguistics: Volume 1

View full text Add to dashboard Cite

Natural language understanding and dialog management are two integral components of interactive dialog systems. Previous research has used machine learning techniques to individually optimize these components, with different forms of direct and indirect supervision. We present an approach to integrate the learning of both a dialog strategy using reinforcement learning, and a semantic parser for robust natural language understanding, using only natural dialog interaction for supervision. Experimental results on a simulated task of robot instruction demonstrate that joint learning of both components improves dialog performance over learning either of these components alone.

show abstract

“…Many data-driven methods have been proposed among which RL-based ones are the most popular. The GASARSA [8] and KTD [9] algorithms are two of the representative methods based on on-line RL. These methods focus on high performance and sample-efficient on-line learning.…”

Section: Related Workmentioning

confidence: 99%

Policy Optimization for Spoken Dialog Management Using Genetic Algorithm

Ren

Zhao

Yan

2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThe optimization of spoken dialog management policies is a non-trivial task due to the erroneous inputs from speech recognition and language understanding modules. The dialog manager needs to ground uncertain semantic information at times to fully understand the need of human users and successfully complete the required dialog tasks. Approaches based on reinforcement learning are currently mainstream in academia and have been proved to be effective, especially when operating in noisy environments. However, in reinforcement learning the dialog strategy is often represented by complex numeric model and thus is incomprehensible to humans. The trained policies are very difficult for dialog system designers to verify or modify, which largely limits the deployment for commercial applications. In this paper we propose a novel framework for optimizing dialog policies specified in human-readable domain language using genetic algorithm. We present learning algorithms using user simulator and real human-machine dialog corpora. Empirical experimental results show that the proposed approach can achieve competitive performance on par with some state-of-the-art reinforcement learning algorithms, while maintaining a comprehensible policy structure. key words: spoken dialog management, spoken dialog system, genetic algorithm

show abstract

A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimization

Cited by 49 publications

References 34 publications

Optimising Turn-Taking Strategies With Reinforcement Learning

Optimising Turn-Taking Strategies With Reinforcement Learning

Integrated Learning of Dialog Strategies and Semantic Parsing

Policy Optimization for Spoken Dialog Management Using Genetic Algorithm

Contact Info

Product

Resources

About