Controlling Output Length in Neural Encoder-Decoders

Abstract

Neural encoder-decoder models have shown great success in many sequence generation tasks. However, previous work has not investigated situations in which we would like to control the length of encoder-decoder outputs. This capability is crucial for applications such as text summarization, in which we have to generate concise summaries with a desired length. In this paper, we propose methods for controlling the output sequence length for neural encoder-decoder models: two decoding-based methods and two learning-based methods.1 Results show that our learning-based methods have the capability to control length without degrading summary quality in a summarization task.

Japanese Sentence Compression with a Large Training Dataset

“…With lstm+leninit, 2 out of 100 sentences, chosen randomly, ended with a word that cannot be located at the end of a sentence. In contrast, with lstm+lenemb, 24 sentences ended with such words and therefore are ungrammatical, although lenemb has shown to be effective in abstractive sentence summarization (Kikuchi et al, 2016). This result suggests that lstm+lenemb is excessively affected by the desired length because lenemb receives the potential desired length at each time of decoding.…”

“…As the second and the third models, we extend the first model to control the output length (Kikuchi et al, 2016). The second model, lstm+leninit, initializes the memory cell of the decoder as follows: m 0 = tarlen * b len where tarlen is the desired output length, and b len is a trainable parameter.…”

Generating Diverse Story Continuations with Controllable Semantics

“…Each control variable corresponds to an attribute of a sentence. Compared to previous work that only seeks to control the values of sentiment (Hu et al, 2017) and length (Kikuchi et al, 2016), we further explore neural text generation with particular verbal predicates, semantic frames, and automatically-induced clusters.…”

Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

“…Thus, we expect to decisively prohibit excessive generation. Finally, we evaluate the effectiveness of our method on well-studied ABS benchmark data provided by Rush et al (2015), and evaluated in (Chopra et al, 2016;Nallapati et al, 2016b;Kikuchi et al, 2016;Takase et al, 2016;Ayana et al, 2016;Gulcehre et al, 2016).…”