Learning dialogue strategies within the Markov decision process framework

Levin, Esther; Pieraccini, Roberto; Eckert, Wieland

doi:10.1109/asru.1997.658989

Cited by 89 publications

(60 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Humans have a greater propensity to criticize what is wrong than to provide positive proposals. In this context, Reinforcement Learning (RL) [1] appears as the best solution to the problem and have been first proposed in [2] and further developed in [3][4] [5]. The main differences between the approaches rely in the way they model the dialogue manager's environment during the learning process.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning for Spoken Dialogue Management: An Experiment with Speech-Based Database Querying

Pietquin

2006

Artificial Intelligence: Methodology, Systems, and Applications

View full text Add to dashboard Cite

Abstract. Although speech and language processing techniques achieved a relative maturity during the last decade, designing a spoken dialogue system is still a tailoring task because of the great variability of factors to take into account. Rapid design and reusability across tasks of previous work is made very difficult. For these reasons, machine learning methods applied to dialogue strategy optimization has become a leading subject of researches since the mid 90's. In this paper, we describe an experiment of reinforcement learning applied to the optimization of speech-based database querying. We will especially emphasize on the sensibility of the method relatively to the dialogue modeling parameters in the framework of the Markov decision processes, namely the state space and the reinforcement signal. The evolution of the design will be exposed as well as results obtained on a simple real application.

show abstract

Section: Introductionmentioning

confidence: 99%

“…The main differences between the approaches rely in the way they model the dialogue manager's environment during the learning process. In [2] and [5], the environment is modeled as a set of independent modules (i.e. ASR system, user) processing information (this approach will be adopted in this paper).…”

Section: Introductionmentioning

confidence: 99%

Machine Learning for Spoken Dialogue Management: An Experiment with Speech-Based Database Querying

Pietquin

2006

Artificial Intelligence: Methodology, Systems, and Applications

View full text Add to dashboard Cite

show abstract

“…The statistical optimization of dialogue management in dialogue systems through Reinforcement Learning (RL) has been an active thread of research for more than two decades (Levin et al, 1997; Lemon and Pietquin, 2007; Laroche et al, 2010; Gašić et al, 2012; Daubigney et al, 2012). Dialogue management has been successfully modelled as a Partially Observable Markov Decision Process (POMDP) (Williams and Young, 2007; Gašić et al, 2012), which leads to systems that can learn from data and which are robust to noise.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Katagiri

Nakano

Fernández

et al. 2016

View full text Add to dashboard Cite

We extend special thanks to our Local co-Chairs, Ron Artstein and Alesia Gainer, and their team of student volunteers. We know SIGDIAL 2016 would not have been possible without Ron and Alesia, who invested so much effort in arranging the conference venue and accommodations, handling registration, making banquet arrangements, and handling numerous other preparations for the conference. The student volunteers for on-site assistance also deserve our appreciation.Ethan Selfridge, Sponsorships Chair, has earned our appreciation for recruiting and liaising with our conference sponsors, many of whom continue to contribute year after year. Sponsorships support valuable aspects of the program, such as the invited speakers and conference banquet. In recognition of this, we gratefully acknowledge the support of our sponsors: (Platinum level) Microsoft Research, Xerox and PARC, Intel, (Gold level) Facebook, (Silver level) Amazon Alexa, Interactions, Educational Testing Service, Honda Research Institute, and Yahoo!. At the same time, we thank Priscilla Rasmussen at the ACL for tirelessly handling the financial aspects of sponsorship for SIGDIAL 2016, and for securing our ISBN.iii We also thank the SIGdial board, especially officers Amanda Stent, Jason Williams and Kristiina Jokinen for their advice and support from beginning to end.Finally, we thank all the authors of the papers in this volume, and all the conference participants for making this stimulating event a valuable opportunity for growth in the research areas of discourse and dialogue. AbstractThis paper presents an end-to-end framework for task-oriented dialog systems using a variant of Deep Recurrent QNetworks (DRQN). The model is able to interface with a relational database and jointly learn policies for both language understanding and dialog strategy. Moreover, we propose a hybrid algorithm that combines the strength of reinforcement learning and supervised learning to achieve faster learning speed. We evaluated the proposed model on a 20 Question Game conversational game simulator. Results show that the proposed method outperforms the modular-based baseline and learns a distributed representation of the latent dialog state. IntroductionTask-oriented dialog systems have been an important branch of spoken dialog system (SDS) research (Raux et al., 2005; Young, 2006; Bohus and Rudnicky, 2003). The SDS agent has to achieve some predefined targets (e.g. booking a flight) through natural language interaction with the users. The typical structure of a task-oriented dialog system is outlined in Figure 1 (Young, 2006). This pipeline consists of several independently-developed modules: natural language understanding (the NLU) maps the user utterances to some semantic representation. This information is further processed by the dialog state tracker (DST), which accumulates the input of the turn along with the dialog history. The DST outputs the current dialog state and the dialog policy selects the next system action based on the dialog state. Then natural language gene...

show abstract

“…The great variability of these factors makes rapid design of dialogue strategies and reusability across tasks of previous work very complex. For these reasons, automatic learning of optimal strategies is currently a leading domain of researches [1] [2][3] [4]. Yet, the low amount of data generally available for learning and testing dialogue strategies does not contain enough information to explore the whole space of dialogue states (and of strategies).…”

Section: Introductionmentioning

confidence: 99%

Learning to Ground in Spoken Dialogue Systems

Pietquin

2007

2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07

View full text Add to dashboard Cite

Machine learning methods such as reinforcement learning applied to dialogue strategy optimization has become a leading subject of researches since the mid 90's. Indeed, the great variability of factors to take into account makes the design of a spoken dialogue system a tailoring task and reusability of previous work is very difficult. Yet, techniques such as reinforcement learning are very demanding in training data while obtaining a substantial amount of data in the particular case of spoken dialogues is time-consuming and therefore expansive. In order to expand existing data sets, dialogue simulation techniques are becoming a standard solution.In this paper, we present a user model for realistic spoken dialogue simulation and a method for using this model so as to simulate the grounding process. This allows including grounding subdialogues as actions in the reinforcement learning process and learning adapted strategy.

show abstract

Learning dialogue strategies within the Markov decision process framework

Cited by 89 publications

References 4 publications

Machine Learning for Spoken Dialogue Management: An Experiment with Speech-Based Database Querying

Machine Learning for Spoken Dialogue Management: An Experiment with Speech-Based Database Querying

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Learning to Ground in Spoken Dialogue Systems

Contact Info

Product

Resources

About