Controllable Text Generation with Focused Variation

Abstract

This work introduces Focused-Variation Network (FVN), a novel model to control language generation. The main problems in previous controlled language generation models range from the difficulty of generating text according to the given attributes, to the lack of diversity of the generated texts. FVN addresses these issues by learning disjoint discrete latent spaces for each attribute inside codebooks, which allows for both controllability and diversity, while at the same time generating fluent text. We evaluate FVN on two text generation datasets with annotated content and style, and show state-of-the-art performance as assessed by automatic and human evaluations.

Semantic Matching for Sequence-to-Sequence Learning

“…Hence, M (X) and M (Ŷ ) are content word sets, extracted based on the POS tags using NLTK. One can employ this prior knowledge to perform different operations for words: (i) for content words, we should encourage partial matching between the sentences by L SEM ; and (ii) for style words, we should discourage matching (Hu et al, 2017). More details are provided in Appendix A.1.…”

“…Experiments are implemented with Tensorflow based on texar . For a fair comparison, we use a similar model configuration to that of (Hu et al, 2017;. One-layer GRU (Cho et al, 2014) encoder and LSTM attention decoder (generator) are used.…”

“…Baselines We implemented CtrlGen (Hu et al, 2017) and LM as our baselines and further added SEPAM on these two models for validation. Further, conditional variational encoder (CVAE) and retrievalbased methods are added as two baselines.…”

Stylistic Chinese Poetry Generation via Unsupervised Style Disentanglement

“…Recently, a few works have been proposed for disentangled representation learning. Info-GAN was proposed for generating continuous image data conditioned on image labels and disentangled text generation (Hu et al, 2017) focused on controllable generation in the semi-supervised setting. Though inspired by them, the motivation and proposed models of our work differ from these methods by a large margin.…”

“…Inspired by previous works on image generation and semi-supervised generation (Hu et al, 2017), we propose to incorporate the concept of mutual information in information theory for style disentanglement. Given two random variables X and Y , the mutual information I(X, Y ) measures "the amount of information" obtained about one random variable given another one 1 .…”