Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables

Liu, Zihan; Shin, Jamin; Xu, Yan; Winata, Genta Indra; Xu, Peng; Madotto, Andrea; Fung, Pascale

doi:10.18653/v1/d19-1129

Cited by 60 publications

(55 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work has also empirically validated that there is still ample room for improvement in the intent detection task especially in low-data regimes. Therefore, similar to recent work (Upadhyay et al, 2018;Khalil et al, 2019;Liu et al, 2019c), we will also investigate how to transfer intent detectors to low-resource target languages in few-shot and zero-shot scenarios. We also plan to extend the models to handle out-of-scope prediction (Larson et al, 2019).…”

Section: Discussionmentioning

confidence: 94%

Efficient Intent Detection with Dual Sentence Encoders

Casanueva¹,

Temčinas²,

Gerz³

et al. 2020

Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI

190

137

View full text Add to dashboard Cite

Building conversational systems in new domains and with added functionality requires resource-efficient models that work under lowdata regimes (i.e., in few-shot setups). Motivated by these requirements, we introduce intent detection methods backed by pretrained dual sentence encoders such as USE and Con-veRT. We demonstrate the usefulness and wide applicability of the proposed intent detectors, showing that: 1) they outperform intent detectors based on fine-tuning the full BERT-Large model or using BERT as a fixed black-box encoder on three diverse intent detection data sets; 2) the gains are especially pronounced in few-shot setups (i.e., with only 10 or 30 annotated examples per intent); 3) our intent detectors can be trained in a matter of minutes on a single CPU; and 4) they are stable across different hyperparameter settings. In hope of facilitating and democratizing research focused on intention detection, we release our code, as well as a new challenging single-domain intent detection dataset comprising 13,083 annotated examples over 77 intents.

show abstract

Section: Discussionmentioning

confidence: 94%

Efficient Intent Detection with Dual Sentence Encoders

Casanueva¹,

Temčinas²,

Gerz³

et al. 2020

Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI

190

137

View full text Add to dashboard Cite

show abstract

“…Recently, several task-oriented dialogue models are introduced to tackle the resource scarcity challenges in target domains (Bapna et al, 2017;Wu et al, 2019a; and target languages (Mrkšić et al, 2017;Schuster et al, 2019;Chen et al, 2018;Liu et al, 2019b), and large pre-trained language models are shown to possess the capability to quickly adapt to taskoriented dialogue tasks by using only a few data samples (Peng et al, 2020b;Madotto et al, 2020b;.…”

Section: Related Workmentioning

confidence: 99%

Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems

Madotto¹,

Cahyawijaya²,

Winata³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

Self Cite

View full text Add to dashboard Cite

Task-oriented dialogue systems are either modularized with separate dialogue state tracking (DST) and management steps or end-to-end trainable. In either case, the knowledge base (KB) plays an essential role in fulfilling user requests. Modularized systems rely on DST to interact with the KB, which is expensive in terms of annotation and inference time. Endto-end systems use the KB directly as input, but they cannot scale when the KB is larger than a few hundred entries. In this paper, we propose a method to embed the KB, of any size, directly into the model parameters. The resulting model does not require any DST or template responses, nor the KB as input, and it can dynamically update its KB via finetuning. We evaluate our solution in five taskoriented dialogue datasets with small, medium, and large KB size. Our experiments show that end-to-end models can effectively embed knowledge bases in their parameters and achieve competitive performance in all evaluated datasets 1 .

show abstract

“…Sachan et al (2018); Jia, Liang, and Zhang (2019) injected target domain knowledge into language models for the fast adaptation, and Jia and Zhang (2020) presented a multi-cell compositional network for NER domain adaptation. Additionally, fast adaptation algorithms have been applied to low-resource languages (Lample and Conneau 2019; Liu et al 2019Liu et al , 2020aWilie et al 2020), accents (Winata et al 2020), and machine translation (Artetxe et al 2018;Lample et al 2018).…”

Section: Related Workmentioning

confidence: 99%

Zero-Resource Cross-Domain Named Entity Recognition

Liu

Winata

Fung

2020

Proceedings of the 5th Workshop on Representation Learning for NLP

Self Cite

View full text Add to dashboard Cite

Cross-domain named entity recognition (NER) models are able to cope with the scarcity issue of NER samples in target domains. However, most of the existing NER benchmarks lack domain-specialized entity types or do not focus on a certain domain, leading to a less effective cross-domain evaluation. To address these obstacles, we introduce a cross-domain NER dataset (CrossNER), a fully-labeled collection of NER data spanning over five diverse domains with specialized entity categories for different domains. Additionally, we also provide a domain-related corpus since using it to continue pre-training language models (domain-adaptive pre-training) is effective for the domain adaptation. We then conduct comprehensive experiments to explore the effectiveness of leveraging different levels of the domain corpus and pre-training strategies to do domain-adaptive pre-training for the crossdomain task. Results show that focusing on the fractional corpus containing domain-specialized entities and utilizing a more challenging pre-training strategy in domain-adaptive pre-training are beneficial for the NER domain adaptation, and our proposed method can consistently outperform existing cross-domain NER baselines. Nevertheless, experiments also illustrate the challenge of this cross-domain NER task. We hope that our dataset and baselines will catalyze research in the NER domain adaptation area. The code and data are available at https://github.com/zliucr/CrossNER.

show abstract

Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables

Cited by 60 publications

References 17 publications

Efficient Intent Detection with Dual Sentence Encoders

Efficient Intent Detection with Dual Sentence Encoders

Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems

Zero-Resource Cross-Domain Named Entity Recognition

Contact Info

Product

Resources

About