MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Budzianowski, Paweł; Wen, Tao; Tseng, Bo-Hsiang; Casanueva, Iñigo; Ultes, Stefan; Ramadan, O.; Gašić, Milica

doi:10.48550/arxiv.1810.00278

Cited by 82 publications

(168 citation statements)

References 0 publications

Supporting

Mentioning

150

Contrasting

Order By: Relevance

“…the performance of different methods on the MultiWoZ dataset[12]. Consistent with intuition, the combination (i.e., PPO-OFF-Comb) of global and local reward reaches highest performance in both match rate (9% improvement) and success rate (8% improvement).…”

supporting

confidence: 71%

“…All models are evaluated on MultiWOZ [12], a multi-domain, multi-intent task-oriented dialog corpus that contains 7 domains, 13 intents, 25 different slots and 10483 dialog sessions. We report the average number of dialog turns, averaging over successful dialog sessions and all dialog sessions respectively, to measure the efficient of accomplishing a task.…”

Section: Dataset and Evaluation Metricmentioning

confidence: 99%

“…For next action prediction, we firstly build the binary classification dataset from MultiWoZ [12] automatically and get 99370, 13157, 13073 labeled samples for training, testing and validation respectively 2 . And then we use BERT [17] as backbone and Adam as optimization algorithm.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…The main contribution of this paper is two-fold: 1) we propose a simple yet effective reward estimator at the action-level to guide the action decision, and 2) We investigate the effects of global reward and local reward, and the experimental results on MultiWOZ [12] show that our methods outperform single global reward about 4.4%, and furthermore, almost 8% when combined with global reward, which indicates the complementary effects of these two types of reward.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Integrating Pretrained Language Model for Dialogue Policy Learning

Wang¹,

Wang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Reinforcement Learning (RL) has been witnessed its potential for training a dialogue policy agent towards maximizing the accumulated rewards given from users. However, the reward can be very sparse for it is usually only provided at the end of a dialog session, which causes unaffordable interaction requirements for an acceptable dialog agent. Distinguished from many efforts dedicated to optimizing the policy and recovering the reward alternatively which suffers from easily getting stuck in local optima and model collapse, we decompose the adversarial training into two steps: 1) we integrate a pre-trained language model as a discriminator to judge whether the current system action is good enough for the last user action (i.e., next action prediction); 2) the discriminator gives and extra local dense reward to guide the agent's exploration. The experimental result demonstrates that our method significantly improves the complete rate ( 4.4%) and success rate ( 8.0%) of the dialogue system.

show abstract

supporting

confidence: 71%

Section: Dataset and Evaluation Metricmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Integrating Pretrained Language Model for Dialogue Policy Learning

Wang¹,

Wang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Generation-based methods in dialogue systems are typically based on sequence-to-sequence modeling. These models are usually trained on a hand-labeled corpus of task-oriented dialogue [3]. Our proposed approach shares elements of both of these methods: it generates questions using a sequence-to-sequence model and stores them in a collection that can be queried using retrieval-based methods.…”

Section: Question Generationmentioning

confidence: 99%

Soliciting User Preferences in Conversational Recommender Systems via Usage-related Questions

Kostric

Balog

Radlinski

2021

Fifteenth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

Conversational Recommender Systems are recommender systems that utilize multi-turn interactions in order to help users find items of interest. Their advantage over traditional, one-shot recommender systems lies in their ability to elicit and adapt to the changing user preference in real time.Common approaches to eliciting user preferences include asking about items and item attributes. This strategies can fail, if the user does not have the prerequisite knowledge about the item or item attributes but they know what they plan to use the item for. In this thesis we propose a novel approach to eliciting preferences by asking implicit questions based on item usage.We identify the sentences form a large corpora of user reviews that contain information about item usage. Based on those sentences and by utilizing crowd workers, we generate questions that could be used in an preference elicitation setting. Lastly, based on the labelled dataset, we train a large neural model to automatically generate question for any viable sentence in the corpus.Using standard metrics for automatic evaluations of generated questions and manual evaluation, we demonstrate the potential viability of such a system in a production setting.Finally, we identify clusters of questions where the system fails. i Thank you to the University of Stavanger for the great years of studying, for giving me the opportunity to do this research and for the access to essential hardware.Thank you Krisztian Balog for being my mentor and a great teacher. Thank you for your contribution in bringing forward this idea and for continuous invaluable guidance throughout the project. Our weekly meetings gave me inspiration and motivation to keep working and expanding the scope of the research.Thank you Filip Radlinski also for being my mentor, supporting the idea and providing great insights and new ideas.Thank you to my girlfriend Kirsti for unwavering support through my years of study, and especially these last couple of months while working with this thesis.Thank you to my daughter Matilde, born just after we started this project. While giving me some sleepless nights and countless of distractions, you bring joy to my every day, even when I am stressed or tired.

show abstract

SUPPLE: A Dialogue Management Approach Based on Conversation Patterns

Kunneman¹,

Hindriks²

2022

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

show abstract

MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Cited by 82 publications

References 0 publications

Integrating Pretrained Language Model for Dialogue Policy Learning

Integrating Pretrained Language Model for Dialogue Policy Learning

Soliciting User Preferences in Conversational Recommender Systems via Usage-related Questions

SUPPLE: A Dialogue Management Approach Based on Conversation Patterns

Contact Info

Product

Resources

About