MOSS: End-to-End Dialog System Framework with Modular Supervision

Liang, Weixin; Tian, Youzhi; Chen, Chengcai; Yu, Zhou

doi:10.48550/arxiv.1909.05528

Cited by 3 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…proposed a hybrid imitation and reinforcement learning method, jointly learned dialogue policy and response generation. Wen et al [2016], Liang et al [2019] trained language understanding, dialogue state tracking, and dialogue policy modules with a shared dialogue encoder.…”

Section: Related Workmentioning

confidence: 99%

A Simple Language Model for Task-Oriented Dialogue

Hosseini-Asl¹,

McCann²,

Wu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Task-oriented dialogue is often decomposed into three tasks: understanding user input, deciding actions, and generating a response. While such decomposition might suggest a dedicated model for each sub-task, we find a simple, unified approach leads to state-of-the-art performance on the MultiWOZ dataset. SimpleTOD is a simple approach to task-oriented dialogue that uses a single causal language model trained on all sub-tasks recast as a single sequence prediction problem. This allows SimpleTOD to fully leverage transfer learning from pre-trained, open domain, causal language models such as GPT-2. SimpleTOD improves over the prior state-of-the-art by 0.49 points in joint goal accuracy for dialogue state tracking. More impressively, SimpleTOD also improves the main metrics used to evaluate action decisions and response generation in an end-to-end setting for task-oriented dialog systems: inform rate by 8.1 points, success rate by 9.7 points, and combined score by 7.2 points.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

A Simple Language Model for Task-Oriented Dialogue

Hosseini-Asl¹,

McCann²,

Wu³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

End-to-End Trainable Non-Collaborative Dialog System

Liu¹,

Qian²,

Shi³

et al. 2019

Preprint

View full text Add to dashboard Cite

End-to-end task-oriented dialog models have achieved promising performance on collaborative tasks where users willingly coordinate with the system to complete a given task. While in non-collaborative settings, for example, negotiation and persuasion, users and systems do not share a common goal. As a result, compared to collaborate tasks, people use social content to build rapport and trust in these non-collaborative settings in order to advance their goals. To handle social content, we introduce a hierarchical intent annotation scheme, which can be generalized to different non-collaborative dialog tasks. Building upon Transfer-Transfo (Wolf et al. 2019), we propose an end-to-end neural network model to generate diverse coherent responses. Our model utilizes intent and semantic slots as the intermediate sentence representation to guide the generation process. In addition, we design a filter to select appropriate responses based on whether these intermediate representations fit the designed task and conversation constraints. Our noncollaborative dialog model guides users to complete the task while simultaneously keeps them engaged. We test our approach on our newly proposed ANTISCAM dataset and an existing PERSUASIONFORGOOD dataset. Both automatic and human evaluations suggest that our model outperforms multiple baselines in these two non-collaborative tasks.

show abstract

Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation

Liang

Zou

2020

Preprint

View full text Add to dashboard Cite

Open Domain dialog system evaluation is one of the most important challenges in dialog research. Existing automatic evaluation metrics, such as BLEU are mostly referencebased. They calculate the difference between the generated response and a limited number of available references. Likert-score based self-reported user rating is widely adopted by social conversational systems, such as Amazon Alexa Prize chatbots. However, selfreported user rating suffers from bias and variance among different users. To alleviate this problem, we formulate dialog evaluation as a comparison task. We also propose an automatic evaluation model CMADE (Comparison Model for Automatic Dialog Evaluation) that automatically cleans self-reported user ratings as it trains on them. Specifically, we first use a self-supervised method to learn better dialog feature representation, and then use KNN and Shapley to remove confusing samples. Our experiments show that CMADE achieves 89.2% accuracy in the dialog comparison task. Our implementation is available at https://github.com/Weixin-Liang/ dialog_evaluation_CMADE.

show abstract

MOSS: End-to-End Dialog System Framework with Modular Supervision

Cited by 3 publications

References 14 publications

A Simple Language Model for Task-Oriented Dialogue

A Simple Language Model for Task-Oriented Dialogue

End-to-End Trainable Non-Collaborative Dialog System

Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation

Contact Info

Product

Resources

About