Group-wise Contrastive Learning for Neural Dialogue Generation

Abstract

Neural dialogue response generation has gained much popularity in recent years. Maximum Likelihood Estimation (MLE) objective is widely adopted in existing dialogue model learning. However, models trained with MLE objective function are plagued by the lowdiversity issue when it comes to the opendomain conversational setting. Inspired by the observation that humans not only learn from the positive signals but also benefit from correcting behaviors of undesirable actions, in this work, we introduce contrastive learning into dialogue generation, where the model explicitly perceives the difference between the well-chosen positive and negative utterances. Specifically, we employ a pretrained baseline model as a reference. During contrastive learning, the target dialogue model is trained to give higher conditional probabilities for the positive samples, and lower conditional probabilities for those negative samples, compared to the reference model. To manage the multimapping relations prevalent in human conversation, we augment contrastive dialogue learning with group-wise dual sampling. Extensive experimental results show that the proposed group-wise contrastive learning framework is suited for training a wide range of neural dialogue generation models with very favorable performance over the baseline training approaches.

Neural Dialogue Generation Methods in Open Domain: A Survey

“…Cai et al [20] introduced contrastive learning into dialogue generation, where the model explicitly perceives the difference between the well-chosen positive and negative utterances.…”

Dialogue Response Generation via Contrastive Latent Representation Learning

“…In our case, the multi-turn dialogue data setup allows us to further utilize the context-response relationship, and conduct hard negative sampling by using context-response matching models. Following (Cai et al, 2020), we consider training a Multi-hop Selector Network (MSN) (Yuan et al, 2019) which provides matching scores between the context and response inputs. Specifically, we construct a dialogue dataset, in which each context input c is paired with one positive response sample x, and multiple randomly sample distrator response samples x j .…”

“…For auto-regressive model, we consider Dialog-GPT (Zhang et al, 2019), which is a GPT-2 based model that is specifically designed for dialogue response generation. For dialogue response generation model that uses contrastive learning, we include group-wise contrastive learning (GCL) (Cai et al, 2020), which conduct CL between target dialogue model and a pretrained reference model. PLATO (Bao et al, 2019) is another model that uses transformer-based model architecture while including a discrete latent variable to tackle the oneto-many mapping problem.…”

“…Such an approach has been used in the computer vision domain (Robinson et al, 2020), guiding a learning method to correct its mistakes more quickly. In our case, we select the negative samples by using a pretrained context-response matching model (Cai et al, 2020). Given a context input, the responses with the top matching scores would be considered as the negative samples, and used for the contrastive objective.…”

Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders

“…Here, the gains are evaluated using reconstruction loss. Finally, inspired from the contrastive learning paradigm (Cai et al, 2020;Chen et al, 2020a,b;Mitrovic et al, 2020), we propose relationship enhancement to increase similarity between the representations of data within the same group, and differentiate the representations of data between different groups.…”