Group-wise Contrastive Learning for Neural Dialogue Generation

Abstract

Neural dialogue response generation has gained much popularity in recent years. Maximum Likelihood Estimation (MLE) objective is widely adopted in existing dialogue model learning. However, models trained with MLE objective function are plagued by the lowdiversity issue when it comes to the opendomain conversational setting. Inspired by the observation that humans not only learn from the positive signals but also benefit from correcting behaviors of undesirable actions, in this work, we introduce contrastive learning into dialogue generation, where the model explicitly perceives the difference between the well-chosen positive and negative utterances. Specifically, we employ a pretrained baseline model as a reference. During contrastive learning, the target dialogue model is trained to give higher conditional probabilities for the positive samples, and lower conditional probabilities for those negative samples, compared to the reference model. To manage the multimapping relations prevailed in human conversation, we augment contrastive dialogue learning with group-wise dual sampling. Extensive experimental results show that the proposed group-wise contrastive learning framework is suited for training a wide range of neural dialogue generation models with very favorable performance over the baseline training approaches.

Contrastive Data and Learning for Natural Language Processing

“…We plan cover the following: Contrastive Data Augmentation for NLP (Shen et al, 2020;Qu et al, 2021); Text Classification (Fang et al, 2020;Kachuee et al, 2020;Suresh and Ong, 2021;Du et al, 2021;Carlsson et al, 2021;Qiu et al, 2021;Klein and Nabi, 2021); Sentence Embeddings Sedghamiz et al, 2021) including Quick-Thought (Logeswaran and Lee, 2018),Sentence-BERT (Reimers and Gurevych, 2019), Info-Sentence BERT (Zhang et al, 2020a), SimCSE (Gao et al, 2021b), DeCLUTR (Giorgi et al, 2020), ConSERT (Yan et al, 2021b), Di-alogueCSE (Liu et al, 2021a). We will also cover discourse analysis (Iter et al, 2020;Kiyomaru and Kurohashi, 2021); Information Extraction (Qin et al, 2020; Machine Translation (Pan et al, 2021;Vamvas and Sennrich, 2021); Question Answering (Karpukhin et al, 2020;You et al, 2021;Yue et al, 2021); Summarization (Duan et al, 2019; including faithfulness (Cao and Wang, 2021), summary evaluation (Wu et al, 2020a), multilingual summarization , and dialogue summarization ; Text Generation (Chai et al, 2021;Lee et al, 2021b) including logicconsistent text generation (Shu et al, 2021), paraphrase generation (Yang et al, 2021a), grammatical error correction (Cao et al, 2021), dialogue generation (Cai et al, 2020), x-ray report generation (Liu et al, 2021b;…”

Learning with Instance Bundles for Reading Comprehension

“…Learning with negative samples has been explored in many natural language tasks, such as dialogue generation (Cai et al, 2020), word embeddings (Mikolov et al, 2013), language modeling (Noji and Takamura, 2020), etc., and computer vision tasks such as image captioning (Dai and Lin, 2017) answering (Yeh and Chen, 2019) and image classification (Hjelm et al, 2018) try to decrease the mutual information between positive and negative samples.…”

Dialogue Response Generation via Contrastive Latent Representation Learning

“…In our case, the multi-turn dialogue data setup allows us to further utilize the context-response relationship, and conduct hard negative sampling by using context-response matching models. Following (Cai et al, 2020), we consider training a Multi-hop Selector Network (MSN) (Yuan et al, 2019) which provides matching scores between the context and response inputs. Specifically, we construct a dialogue dataset, in which each context input c is paired with one positive response sample x, and multiple randomly sample distrator response samples x j .…”

“…For auto-regressive model, we consider Dialog-GPT (Zhang et al, 2019), which is a GPT-2 based model that is specifically designed for dialogue response generation. For dialogue response generation model that uses contrastive learning, we include group-wise contrastive learning (GCL) (Cai et al, 2020), which conduct CL between target dialogue model and a pretrained reference model. PLATO (Bao et al, 2019) is another model that uses transformer-based model architecture while including a discrete latent variable to tackle the oneto-many mapping problem.…”

“…Such an approach has been used in the computer vision domain (Robinson et al, 2020), guiding a learning method to correct its mistakes more quickly. In our case, we select the negative samples by using a pretrained context-response matching model (Cai et al, 2020). Given a context input, the responses with the top matching scores would be considered as the negative samples, and used for the contrastive objective.…”