CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset

Abstract

To advance multi-domain (cross-domain) dialogue modeling as well as alleviate the shortage of Chinese task-oriented datasets, we propose CrossWOZ, the first large-scale Chinese Cross-Domain Wizard-of-Oz task-oriented dataset. It contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. Moreover, the corpus contains rich annotation of dialogue states and dialogue acts on both user and system sides. About 60% of the dialogues have cross-domain user goals that favor inter-domain dependency and encourage natural transition across domains in conversation. We also provide a user simulator and several benchmark models for pipelined task-oriented dialogue systems, which will facilitate researchers to compare and evaluate their models on this corpus. The large size and rich annotation of CrossWOZ make it suitable to investigate a variety of tasks in cross-domain dialogue modeling, such as dialogue state tracking, policy learning, user simulation, etc.

Recent advances and challenges in task-oriented dialog systems

“…These datasets have higher language variation and task complexity. While most datasets are in English, Zhu et al [72] propose the first large-scale Chinese task-oriented dataset with rich annotations to facilitate the research of Chinese and cross-lingual dialog modeling. An incomplete survey on these dialog datasets is presented in Table 2.…”

RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling

“…ATIS (Hemphill et al, 1990), WOZ 2.0 (Wen et al, 2017), FRAMES (El Asri et al, 2017) and KVRET (Eric et al, 2017) are small-scale datasets built in this way. In contrast, MultiWOZ Budzianowski et al (2018) and Cross-WOZ (Zhu et al, 2020) are two large-scale H2H datasets proposed recently.…”

“…In recent years, we have witnessed that a variety of datasets tailored for task-oriented dialogue have been constructed, such as MultiWOZ (Budzianowski et al, 2018), SGD (Rastogi et al, 2019a) and CrossWOZ (Zhu et al, 2020), along with the increasing interest in conversational AI in both academia and industry (Gao et al, 2018). These datasets have triggered extensive research in either end-to-end or traditional modular taskoriented dialogue modeling (Wen et al, 2019;Dai et al, 2020).…”

“…MultiWOZ (Budzianowski et al, 2018), probably the most promising and notable dialogue corpus collected in a Wizard-of-Oz (i.e., Human-to-Human) way recently, is one order of magnitude larger than the aforementioned corpora collected in the same way. However, it contains noisy systemside state annotations and lacks user-side dialogue acts 2 (Eric et al, 2019;Zhu et al, 2020). Yet another very recent dataset CrossWOZ (Zhu et al, 2020), the first large-scale Chinese H2H dataset for task-oriented dialogue, provides semantic annotations on both user and system side although it is relatively smaller than MultiWOZ.…”

“…However, it contains noisy systemside state annotations and lacks user-side dialogue acts 2 (Eric et al, 2019;Zhu et al, 2020). Yet another very recent dataset CrossWOZ (Zhu et al, 2020), the first large-scale Chinese H2H dataset for task-oriented dialogue, provides semantic annotations on both user and system side although it is relatively smaller than MultiWOZ. The number of domains in both MultiWOZ and CrossWOZ is fewer than 10.…”