C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling

Abstract

Slot filling, a fundamental module of spoken language understanding, often suffers from insufficient quantity and diversity of training data. To remedy this, we propose a novel Cluster-to-Cluster generation framework for Data Augmentation (DA), named C2C-GenDA. It enlarges the training set by reconstructing existing utterances into alternative expressions while keeping semantic. Different from previous DA works that reconstruct utterances one by one independently, C2C-GenDA jointly encodes multiple existing utterances of the same semantics and simultaneously decodes multiple unseen expressions. Jointly generating multiple new utterances allows to consider the relations between generated instances and encourages diversity. Besides, encoding multiple existing utterances endows C2C with a wider view of existing expressions, helping to reduce generation that duplicates existing data. Experiments on ATIS and Snips datasets show that instances augmented by C2C-GenDA improve slot filling by 7.99 (11.9%↑) and 5.76 (13.6%↑) F-scores respectively, when there are only hundreds of training utterances. 1 * Equal contributions.

Generative Conversational Networks

“…We use the original train / validation / test splits provided with each dataset. For Restaurants8k, we randomly split the training set into training (80%) and SNIPS-3 PROTODA 0.881 GCN-RL 0.822 GCN+RL 0.926 (Hou et al, 2020b) and SC-GPT (Peng et al, 2020b) on few-shot intent detection. We allow our learners to train for 5000 iterations.…”

“…C2C-GenDA (cluster to cluster generation for data augmentation) (Hou et al, 2020b) is a generative data augmentation approach focused on slot filling. This method jointly encodes multiple realisations (i.e.…”

“…In Table 4, we show a comparison with C2C-GenDA (Hou et al, 2020b) and SC-GPT (Peng et al, 2020b) on SNIPS. GCN outperforms C2C-GenDA while SC-GPT performs better than GCN, which is expected since it is based on GPT-2 (instead of distilGPT2) and fine-tuned on 400K additional dialogue act -utterance pairs.…”

Generative Conversational Networks

“…In Table 4, we show a comparison with C2C-GenDA (Hou et al, 2020b) and SC-GPT (Peng et al, 2020b) on SNIPS. GCN outperforms C2C-GenDA while SC-GPT performs better than GCN, which is expected since it is based on GPT-2 (instead of distilGPT2) and fine-tuned 400K additional dialogue act -utterance pairs.…”