A Differentiable Generative Adversarial Network for Open Domain Dialogue

Abstract

This work presents a novel methodology to train open domain neural dialogue systems within the framework of Generative Adversarial Networks with gradient based optimization methods. We avoid the non-differentiability related to textgenerating networks approximating the word vector corresponding to each generated token via a top-k softmax. We show that a weighted average of the word vectors of the most probable tokens computed from the probabilities resulting of the top-k softmax leads to a good approximation of the word vector of the generated token. Finally we demonstrate through a human evaluation process that training a neural dialogue system via adversarial learning with this method successfully discourages it from producing generic responses. Instead it tends to produce more informative and variate ones.

Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers

“…[7] proposed a soccer dialogue dataset along with a KG-Copy mechanism for non-goal oriented dialogues which are KG-integrated. In a slightly similar research line, in past years, we also notice the use of variational autoencoders (VAE) [40,17] and generative adversarial networks (GANs) [23,18] in dialogue generation. However, knowledge graph based dialogue generation is not well-explored in these approaches.…”

A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue Structure

“…Decoding details: Neural dialogue systems have been well known to generate too generic and repetitive. This problem has been tackled with many approaches, such as modifying the loss function [52] or using adversarial training [54,63]. Lately, making use of a proper decoding procedure has proved to be essential for generative models to produce good quality non-generic responses [34,49].…”

III. Ikergazte. Nazioarteko ikerketa euskaraz. Kongresuko artikulu bilduma. Ingeniaritza eta Arkitektura

“…Amaitzeko, lan honekin proposatu berri dugun (López Zorrilla et al, 2019) eta testuarekin era guztiz diferentzialean lan egin dezakeen sare sortzaile aurkarien arkitektura baliozkotzen dugu, elkarrizketa sistema automatikoak euskaraz eraikitzeko aproposa dela egiaztatuz.…”

“…Ingelesez corpusaren tamaina handiagoa denez, sareak ere handiagoak dira eta iterazio gehiagotan entrenatu dugu. Zehaztasunak(López Zorrilla et al, 2019) erreferentzian ematen dira.…”