Fˆ2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax

Abstract

Despite recent advances in neural text generation, encoding the rich diversity in human language remains elusive. We argue that the sub-optimal text generation is mainly attributable to the imbalanced token distribution, which particularly misdirects the learning model when trained with the maximumlikelihood objective. As a simple yet effective remedy, we propose two novel methods, F 2 -Softmax and MefMax, for a balanced training even with the skewed frequency distribution. MefMax assigns tokens uniquely to frequency classes, trying to group tokens with similar frequencies and equalize frequency mass between the classes. F 2 -Softmax then decomposes a probability distribution of the target token into a product of two conditional probabilities of (i) frequency class, and (ii) token from the target frequency class. Models learn more uniform probability distributions because they are confined to subsets of vocabularies. Significant performance gains on seven relevant metrics suggest the supremacy of our approach in improving not only the diversity but also the quality of generated texts.

Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling

“…As a standard approach to training a neural text generation model, MLE has been proved to be defective. Choi et al (2020) have shown that MLE may mislead the model because of the imbalanced token distribution. Thus, they design a greedy approach MefMax and factorize Softmax to ensure a balanced training according to the word frequency.…”

“…We performed experiments on the Wikitext-103 1 dataset (Merity et al, 2017), a large-scale benchmark containing more than 29 thousand Wikipedia articles with over 100 million words in total. Wikitext-103 has been widely used in many language modeling models (Welleck et al, 2020;Martins et al, 2020;Choi et al, 2020), but in order to train our POS guided Softmax, we need the corresponding POS tags. We use the Stanford CoreNLP's POS tagger 2 (Manning et al, 2014) to annotate words in Wikitext-103 with XPOS 3 tags (Hornby et al, 2017).…”

“…Holtzman et al (2020) think that MLE can not adequately capture the rich diversity and expression in human language. Choi et al (2020) argue that the imbalanced token distribution inherent in natural language even worsens the low-diversity problem. Based on these analysis, many training-based methods have been proposed.…”

“…Jiang et al (2019) propose to utilize dynamically scaling losses conditioned on the token frequency in the training phase. Choi et al (2020) factorize the probability distribution and design an elaborate token cluster algorithm for a balanced training.…”

Conversational Information Seeking

“…Thus, the models often suffer from generating diverse outputs. This has been addressed using different techniques, such as unlikelihood training (Welleck et al, 2020) and F 2 -Softmax (Choi et al, 2020). Clarification utility maximization (next subsection) also implicitly addresses this issue.…”