Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Abstract

In task-oriented dialogue (ToD), a user holds a conversation with an artificial agent with the aim of completing a concrete task. Although this technology represents one of the central objectives of AI and has been the focus of ever more intense research and development efforts, it is currently limited to a few narrow domains (e.g., food ordering, ticket booking) and a handful of languages (e.g., English, Chinese). This work provides an extensive overview of existing methods and resources in multilingual ToD as an entry point to this exciting and emerging field. We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation. In fact, acquiring annotations or human feedback for each component of modular systems or for data-hungry end-to-end systems is expensive and tedious. Hence, state-of-the-art approaches to multilingual ToD mostly rely on (zero- or few-shot) cross-lingual transfer from resource-rich languages (almost exclusively English), either by means of (i) machine translation or (ii) multilingual representations. These approaches are currently viable only for typologically similar languages and languages with parallel / monolingual corpora available. On the other hand, their effectiveness beyond these boundaries is doubtful or hard to assess due to the lack of linguistically diverse benchmarks (especially for natural language generation and end-to-end evaluation). To overcome this limitation, we draw parallels between components of the ToD pipeline and other NLP tasks, which can inspire solutions for learning in low-resource scenarios. Finally, we list additional challenges that multilinguality poses for related areas (such as speech, fluency in generated text, and human-centred evaluation), and indicate future directions that hold promise to further expand language coverage and dialogue capabilities of current ToD systems.

Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog

“…While earlier datasets focused on a single domain (Henderson et al, 2014a,b;Wen et al, 2017), the focus shifted towards the more realistic multi-domain task-oriented dialogs with the creation of the Mul-tiWOZ dataset , which has been refined and improved in several iterations Zang et al, 2020;Han et al, 2021). Due to the particularly high costs of creating TOD datasets (in comparison with other language understanding tasks) (Razumovskaia et al, 2021), only a handful of monolingual TOD datasets Table 5: Per-language few-shot transfer performance (sample efficiency results) on DST and RR for the baseline TOD-XLMR and the best specialized model (TLM+RS-Mono on OS). in languages other than English (Zhu et al, 2020) or bilingual TOD datasets have been created (Gunasekara et al, 2020;Lin et al, 2021).…”

“…This lack can be attributed to the fact that creating TOD datasets for new languages from scratch or via translation of English datasets is significantly more expensive and time-consuming than for most other NLP tasks. However, the absence of multilingual datasets that are comparable (i.e., aligned) across languages prevents a reliable estimate of effectiveness of cross-lingual transfer techniques in multi-domain TOD (Razumovskaia et al, 2021).…”

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

“…Fine-tuning a large multilingual LM has become a standard for multilingual NLU (Zhang et al, 2019;Kulshreshtha et al, 2020). However, the excessively high data annotation costs for multiple domains and languages still hinder progress in multilingual dialogue (Razumovskaia et al, 2021). In this paper, unlike prior work, we propose to use external unannotated data to mine and automatically label in-domain in-language examples which aid learning in low-data regimes across multiple languages.…”

“…At the same time, porting an NLU system to any new domain and language requires collecting a large indomain dataset, and training a model for the target language . Such in-domain annotations in multiple languages are extremely expensive and time-consuming (Rastogi et al, 2020), also reflected in the fact that large enough dialogue NLU datasets for other languages are still few and far between (Razumovskaia et al, 2021). This in turn creates the demand for strong multilingual and crosslingual methods which generalise well and learn effectively in zero-shot and few-shot scenarios.…”

GL-CLeF: A Global–Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding

“…This allows mBERT to share embeddings across languages, which achieves promising performance on various cross-lingual NLP tasks; 2) Ensemble-Net. Razumovskaia et al (2021) propose an Ensemble-Net where predictions are determined by 8 independent models through majority voting, each separately trained on a single source language, which achieves promising performance on zero-shot cross-lingual SLU; 3) AR-S2S-PTR. Rongali et al (2020) proposed a unified sequence-to-sequence models with pointer generator network for cross-lingual SLU; 4) IT-S2S-PTR.…”