Controllable Abstractive Summarization

Fan, Angela; Grangier, David; Auli, Michael

doi:10.18653/v1/w18-2706

Cited by 217 publications

(211 citation statements)

References 22 publications

Supporting

Mentioning

200

Contrasting

Order By: Relevance

“…Very few works explicitly define a bernoulli distribution for the selector, then train with the REINFORCE algorithm (Ling and Rush, 2017;Chen and Bansal, 2018), but the selection targets at a high recall regardless of the low precision, so the controllability over generated text is weak. Fan et al (2018) control the generation by manually concatenating entity embeddings, while our model is much more flexible by explicitly defining the selection probability over all source tokens.…”

Section: Related Workmentioning

confidence: 99%

Select and Attend: Towards Controllable Content Selection in Text Generation

Shen¹,

Suzuki

Inui

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Many text generation tasks naturally contain two steps: content selection and surface realization. Current neural encoder-decoder models conflate both steps into a black-box architecture. As a result, the content to be described in the text cannot be explicitly controlled. This paper tackles this problem by decoupling content selection from the decoder. The decoupled content selection is human interpretable, whose value can be manually manipulated to control the content of generated text. The model can be trained end-to-end without human annotations by maximizing a lower bound of the marginal likelihood. We further propose an effective way to trade-off between performance and controllability with a single adjustable hyperparameter. In both data-to-text and headline generation tasks, our model achieves promising results, paving the way for controllable content selection in text generation.

show abstract

Section: Related Workmentioning

confidence: 99%

Select and Attend: Towards Controllable Content Selection in Text Generation

Shen¹,

Suzuki

Inui

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…These systems can be used in various application areas, such as text summarization (Fan et al, 2018), adversarial example generation (Iyyer et al, 2018), dialogue (Niu and Bansal, 2018), and data-to-document generation (Wiseman et al, 2018). However, prior work on controlled generation has typically assumed a known, finite set of values that the controlled attribute can take on.…”

Section: Introductionmentioning

confidence: 99%

Controllable Paraphrase Generation with a Syntactic Exemplar

Chen

Tang

Wiseman

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Prior work on controllable text generation usually assumes that the controlled attribute can take on one of a small set of values known a priori. In this work, we propose a novel task, where the syntax of a generated sentence is controlled rather by a sentential exemplar. To evaluate quantitatively with standard metrics, we create a novel dataset with human annotations. We also develop a variational model with a neural module specifically designed for capturing syntactic knowledge and several multitask training objectives to promote disentangled representation learning. Empirically, the proposed model is observed to achieve improvements over baselines and learn to capture desirable characteristics. 1 arXiv:1906.00565v1 [cs.CL] 3 Jun 2019Encoders. At test time, we want to have different combinations of semantic and syntactic inputs, which naturally suggests separate parameterizations for q φ (y|x) and q φ (z|x). Specifically, q φ (y|x) is parameterized by a word averaging encoder followed by a three-layer feedforward neural network since it has been observed that word averaging encoders perform surprisingly well for semantic tasks (Wieting et al., 2016). q φ (z|x) is parameterized by a bidirectional long short-term memory network (LSTM; Hochreiter and Schmidhuber, 1997) also followed by a three-layer feedforward neural network, where we concatenate the forward and backward vectors produced by the biLSTM and then take the average of these vectors.Decoders. As shown in Figure 3, at each time step, we concatenate the syntactic variable z with the previous word's embedding as the input to the

show abstract

“…To investigate whether the proposed method can generate good headlines for unseen lengths, we excluded headlines whose lengths are equal to the desired length (len) from the training data. The len = 10 len = 13 len = 26 32.85 11.78 28.52 Previous studies for controlling output length Kikuchi et al (2016) 26.73 8.39 23.88 Fan et al (2018) 30.00 10.27 26.43 Other previous studies Rush et al (2015) 28.18 8.49 23.81 Nagata (2017) 32.28 10.54 27.80 Zhou et al (2017) 29.21 9.56 25.51 Li et al (2017) 31.79 10.75 27.48 Li et al (2018) 29.33 10.24 25.24 Table 3 shows the recall-oriented ROUGE scores on the DUC-2004 test set. Following the evaluation protocol (Over et al, 2007), we truncated characters over 75 bytes.…”

Section: Resultsmentioning

confidence: 99%

“…In fact, Figure 1 shows a large variance in output sequences produced by a widely used encoder-decoder model (Luong et al, 2015), which has no mechanism for controlling the length of the output sequences. Fan et al (2018) trained embeddings that correspond to each output length to control the output sequence length. Since the embeddings for different lengths are independent, it is hard to generate a sequence of the length that is infrequent in training data.…”

Section: Introductionmentioning

confidence: 99%

Positional Encoding to Control Output Sequence Length

Takase¹,

Okazaki²

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Neural encoder-decoder models have been successful in natural language generation tasks. However, real applications of abstractive summarization must consider additional constraint that a generated summary should not exceed a desired length. In this paper, we propose a simple but effective extension of a sinusoidal positional encoding (Vaswani et al., 2017) to enable neural encoder-decoder model to preserves the length constraint. Unlike in previous studies where that learn embeddings representing each length, the proposed method can generate a text of any length even if the target length is not present in training data. The experimental results show that the proposed method can not only control the generation length but also improve the ROUGE scores.

show abstract

Controllable Abstractive Summarization

Cited by 217 publications

References 22 publications

Select and Attend: Towards Controllable Content Selection in Text Generation

Select and Attend: Towards Controllable Content Selection in Text Generation

Controllable Paraphrase Generation with a Syntactic Exemplar

Positional Encoding to Control Output Sequence Length

Contact Info

Product

Resources

About