Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking

Dai, Yinpei; Li, Hangyu; Li, Yongbin; Sun, Jian; Huang, Fei; Si, Luo; Zhu, Xiaodan

doi:10.18653/v1/2021.acl-short.111

Cited by 21 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another benefit of using pretrained GPT2 is faster training time as we observed the VDTN+GPT2 converged much earlier than training it from scratch. From these observations, we are excited to see more future extension of SOTA unimodal DST models (Lin et al, 2021;Dai et al, 2021) and large pretrained LMs (Brown et al, 2020;Raffel et al, 2020), especially ones with multimodal learning such as (Lu et al, 2019;, to MM-DST task.…”

Section: Resultsmentioning

confidence: 99%

Multimodal Dialogue State Tracking

Lê¹,

Chen²,

Hoi³

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Designed for tracking user goals in dialogues, a dialogue state tracker is an essential component in a dialogue system. However, the research of dialogue state tracking has largely been limited to unimodality, in which slots and slot values are limited by knowledge domains (e.g. restaurant domain with slots of restaurant name and price range) and are defined by specific database schema. In this paper, we propose to extend the definition of dialogue state tracking to multimodality. Specifically, we introduce a novel dialogue state tracking task to track the information of visual objects that are mentioned in video-grounded dialogues. Each new dialogue utterance may introduce a new video segment, new visual objects, or new object attributes and a state tracker is required to update these information slots accordingly. We created a new synthetic benchmark and designed a novel baseline, Video-Dialogue Transformer Network (VDTN), for this task. VDTN combines both object-level features and segment-level features and learns contextual dependencies between videos and dialogues to generate multimodal dialogue states. We optimized VDTN for a state generation task as well as a self-supervised video understanding task which recovers video segment or object representations. Finally, we trained VDTN to use the decoded states in a response prediction task. Together with comprehensive ablation and qualitative analysis, we discovered interesting insights towards building more capable multimodal dialogue systems. Dialogue HUMAN: I am looking for a cheap restaurant in the centre of the city.Dialogue State PRICE = cheap AREA = centre Database NAME PRICE AREA FOOD CHOICE Pizza centre cheap centre italian JP house cheap centre japanese Pizza express moderate west italian Rice house expensive east chinese

show abstract

Section: Resultsmentioning

confidence: 99%

Multimodal Dialogue State Tracking

Lê¹,

Chen²,

Hoi³

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…Traditional dialogue systems [17,33] usually consist of three components: natural language understanding (NLU) [28,30,58,59], dialogue management (DM) [6,7,18], and natural language generation (NLG) [50,63,65,66] modules. Empirically, NLU plays the most important role in task-oriented dialogue systems, including tasks such as intent detection [12,13,29,57], slot filling [61], and semantic parsing [19?…”

Section: Related Workmentioning

confidence: 99%

Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue System

Lin,

Wu,

Huang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we present Duplex Conversation, a multi-turn, multimodal spoken dialogue system that enables telephone-based agents to interact with customers like a human. We use the concept of full-duplex in telecommunication to demonstrate what a humanlike interactive experience should be and how to achieve smooth turn-taking through three subtasks: user state detection, backchannel selection, and barge-in detection. Besides, we propose semisupervised learning with multimodal data augmentation to leverage unlabeled data to increase model generalization. Experimental results on three sub-tasks show that the proposed method achieves consistent improvements compared with baselines. We deploy the Duplex Conversation to Alibaba intelligent customer service and share lessons learned in production. Online A/B experiments show that the proposed system can significantly reduce response latency by 50%. CCS CONCEPTS• Computing methodologies → Discourse, dialogue and pragmatics; • Information systems → Multimedia information systems.

show abstract

“…Broadly, multi-domain learning (Joshi et al, 2012) includes training and evaluation using data from multiple domains (e.g. Dredze and Crammer (2008); Qin et al (2020); Dai et al (2021b)). Sometimes, it is assumed that the input texts are accompanied by a domain label (e.g.…”

Section: Introductionmentioning

confidence: 99%

Multi-Domain Targeted Sentiment Analysis

Toledo-Ronen¹,

Orbach²,

Katz³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Targeted Sentiment Analysis (TSA) is a central task for generating insights from consumer reviews. Such content is extremely diverse, with sites like Amazon or Yelp containing reviews on products and businesses from many different domains. A real-world TSA system should gracefully handle that diversity. This can be achieved by a multi-domain model -one that is robust to the domain of the analyzed texts, and performs well on various domains. To address this scenario, we present a multi-domain TSA system based on augmenting a given training set with diverse weak labels from assorted domains. These are obtained through self-training on the YELP reviews corpus. Extensive experiments with our approach on three evaluation datasets across different domains demonstrate the effectiveness of our solution. We further analyze how restrictions imposed on the available labeled data affect the performance, and compare the proposed method to the costly alternative of manually gathering diverse TSA labeled data. Our results and analysis show that our approach is a promising step towards a practical domain-robust TSA system.

show abstract

Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking

Cited by 21 publications

References 30 publications

Multimodal Dialogue State Tracking

Multimodal Dialogue State Tracking

Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue System

Multi-Domain Targeted Sentiment Analysis

Contact Info

Product

Resources

About