Multi-domain dialogue success classifiers for policy training

Vandyke, David; Su, Pei-Hao; Gašić, Milica; Mrkšić, Nikola; Wen, Tao; Young, Steve

doi:10.1109/asru.2015.7404865

Cited by 15 publications

(24 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted here that [20] followed a different approach, utilising a simulated user and allowing their model access to turn-by-turn returns; this type of feedback is not possible in our case since we are using un-annotated spoken dialogue data. Regardless of this, [20] propose three feature sets containing information about the user's dialogue act, the system's dialogue act, current turn number and belief state information. The major difference among the three sets lies in the belief state information: a) F is defined as the full belief, b) F28 contains no belief state information, and c) F74 contains the entropy of each slot in the belief state.…”

Section: Input Featuresmentioning

confidence: 99%

Section: Input Featuresmentioning

confidence: 99%

“…We here briefly describe the features proposed in [20] that we used as a benchmark for our system. It should be noted here that [20] followed a different approach, utilising a simulated user and allowing their model access to turn-by-turn returns; this type of feedback is not possible in our case since we are using un-annotated spoken dialogue data. Regardless of this, [20] propose three feature sets containing information about the user's dialogue act, the system's dialogue act, current turn number and belief state information.…”

Section: Input Featuresmentioning

confidence: 99%

“…[18] propose a reward inference scheme, called Task Completion Transfer Learning, to estimate dialogue quality on-line and evaluate it on simulated dialogues. In order to classify a dialogue as successful or not in dialogues with humans, [19] and [20] propose to use Recurrent Neural Networks trained with an agenda-based simulated user. This approach was shown to have potential to generalise to interactions with human users, and since it is the closest work to our approach, we here regard it as the state of the art.…”

Section: Prior Workmentioning

confidence: 99%

See 3 more Smart Citations

Predicting dialogue success, naturalness, and length with acoustic features

Papangelis

Kotti

Stylianou

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Statistical methods for Spoken Dialogue Systems have been shown to reduce the cost of development, while successfully handling a variety of applications. However, such systems are usually trained with simulated users or paid subjects in controlled settings. While this may be sufficient to jump-start learning in the various subcomponents, learning is very much dependent on the complete knowledge that we have about the interaction. Relatively few works have focused on this problem, and we here propose to extract lowlevel audio descriptors and use them as input to various classifiers, namely support vector machines, Gaussian process regressors, and random forests, to predict metrics that are constituents of user satisfaction from acoustic features. While our approach is not directly comparable to the current state of the art, results show that models using the proposed feature set outperform models that use state of the art features extracted from the belief state.

show abstract

Section: Input Featuresmentioning

confidence: 99%

Section: Input Featuresmentioning

confidence: 99%

Section: Input Featuresmentioning

confidence: 99%

Section: Prior Workmentioning

confidence: 99%

See 2 more Smart Citations

Predicting dialogue success, naturalness, and length with acoustic features

Papangelis

Kotti

Stylianou

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…The SDS used to collect this corpus was produced by VocalIQ 7 (Mrkšić et al, 2015). Users were again recruited via AMT, but interacted with this SDS via microphone using the Chrome browser.…”

Section: Corpora Creationmentioning

confidence: 99%

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Katagiri

Nakano

Fernández

et al. 2016

View full text Add to dashboard Cite

We extend special thanks to our Local co-Chairs, Ron Artstein and Alesia Gainer, and their team of student volunteers. We know SIGDIAL 2016 would not have been possible without Ron and Alesia, who invested so much effort in arranging the conference venue and accommodations, handling registration, making banquet arrangements, and handling numerous other preparations for the conference. The student volunteers for on-site assistance also deserve our appreciation.Ethan Selfridge, Sponsorships Chair, has earned our appreciation for recruiting and liaising with our conference sponsors, many of whom continue to contribute year after year. Sponsorships support valuable aspects of the program, such as the invited speakers and conference banquet. In recognition of this, we gratefully acknowledge the support of our sponsors: (Platinum level) Microsoft Research, Xerox and PARC, Intel, (Gold level) Facebook, (Silver level) Amazon Alexa, Interactions, Educational Testing Service, Honda Research Institute, and Yahoo!. At the same time, we thank Priscilla Rasmussen at the ACL for tirelessly handling the financial aspects of sponsorship for SIGDIAL 2016, and for securing our ISBN.iii We also thank the SIGdial board, especially officers Amanda Stent, Jason Williams and Kristiina Jokinen for their advice and support from beginning to end.Finally, we thank all the authors of the papers in this volume, and all the conference participants for making this stimulating event a valuable opportunity for growth in the research areas of discourse and dialogue. AbstractThis paper presents an end-to-end framework for task-oriented dialog systems using a variant of Deep Recurrent QNetworks (DRQN). The model is able to interface with a relational database and jointly learn policies for both language understanding and dialog strategy. Moreover, we propose a hybrid algorithm that combines the strength of reinforcement learning and supervised learning to achieve faster learning speed. We evaluated the proposed model on a 20 Question Game conversational game simulator. Results show that the proposed method outperforms the modular-based baseline and learns a distributed representation of the latent dialog state. IntroductionTask-oriented dialog systems have been an important branch of spoken dialog system (SDS) research (Raux et al., 2005; Young, 2006; Bohus and Rudnicky, 2003). The SDS agent has to achieve some predefined targets (e.g. booking a flight) through natural language interaction with the users. The typical structure of a task-oriented dialog system is outlined in Figure 1 (Young, 2006). This pipeline consists of several independently-developed modules: natural language understanding (the NLU) maps the user utterances to some semantic representation. This information is further processed by the dialog state tracker (DST), which accumulates the input of the turn along with the dialog history. The DST outputs the current dialog state and the dialog policy selects the next system action based on the dialog state. Then natural language gene...

show abstract

Building multi-domain conversational systems from single domain resources

Griol

Molina

2018

Neurocomputing

View full text Add to dashboard Cite

Current advances in the development of mobile and smart devices have generated a growing demand for natural human-machine interaction and favored the intelligent assistant metaphor, in which a single interface gives access to a wide range of functionalities and services. Conversational systems constitute and important enabling technology in this paradigm. However, they are usually defined to interact in semanticrestricted domains in which users are offered a limited number of options and functionalities. The design of multi-domain systems implies that a single conversational system is able to assist the user in a variety of tasks. In this paper we propose an architecture for the development of multi-domain conversational systems that allows: (1) integrating available multi and single domain speech recognition and understanding modules, (2) combining available system in the different domains implied so that it is not necessary to generate new expensive resources for the multidomain system, (3) achieving better domain recognition rates to select the appropriate interaction management strategies. We have evaluated our proposal combining three systems in different domains to show that the proposed architecture can satisfactory deal with multimodain dialogs.

show abstract

Multi-domain dialogue success classifiers for policy training

Cited by 15 publications

References 30 publications

Predicting dialogue success, naturalness, and length with acoustic features

Predicting dialogue success, naturalness, and length with acoustic features

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Building multi-domain conversational systems from single domain resources

Contact Info

Product

Resources

About