Abstractive Document Summarization with a Graph-Based Attentional Neural Model

Abstract

Abstractive summarization is the ultimate goal of document summarization research, but previously it is less investigated due to the immaturity of text generation techniques. Recently impressive progress has been made to abstractive sentence summarization using neural models. Unfortunately, attempts on abstractive document summarization are still in a primitive stage, and the evaluation results are worse than extractive methods on benchmark datasets. In this paper, we review the difficulties of neural abstractive document summarization, and propose a novel graph-based attention mechanism in the sequence-to-sequence framework. The intuition is to address the saliency factor of summarization, which has been overlooked by prior works. Experimental results demonstrate our model is able to achieve considerable improvement over previous neural abstractive models. The data-driven neural abstractive method is also competitive with state-of-the-art extractive methods.

Deep Communicating Agents for Abstractive Summarization

“…1 We select the MLE models with the lowest negative log-likelihood and the MLE+RL models with the highest ROUGE-L scores on a sample of validation data to evaluate on the test Model ROUGE-1 ROUGE-2 ROUGE-L SummaRuNNer (Nallapati et al, 2017) 39.60 16.20 35.30 graph-based attention (Tan et al, 2017) 38.01 13.90 34.00 pointer generator (See et al, 2017) 36.44 15.66 33.42 pointer generator + coverage (See et al, 2017) 39.53 17.28 36.38 controlled summarization with fixed values (Fan et al, 2017) 39.75 17.29 36.54 RL, with intra-attention (Paulus et al, 2018) 41.16 15.75 39.08 ML+RL, with intra-attention (Paulus et al, 2018) 39 Model Rouge-1 Rouge-2 Rouge-L ML, no intra-attention (Paulus et al, 2018) 44.26 27.43 40.41 RL, no intra-attention (Paulus et al, 2018) 47.22 30.51 43.27 ML+RL, no intra-attention (Paulus et al, 2018) 47 set. At test time, we use beam search of width 5 on all our models to generate final predictions.…”

Guiding Extractive Summarization with Question-Answering Rewards

“…Distraction-M3 (Chen et al, 2016b) trains the summarization system to distract its attention to traverse different regions of the source article. Graph attention (Tan et al, 2017) introduces a graph-based attention mechanism to enhance the encoderdecoder framework. PointerGen+Cov.…”

An Effective Joint Framework for Document Summarization

“…The value of k, which denotes the number of sentences being selected as the basis of abstractive summarization process, is set to 5. We compare our model with two state-of-the-art approaches, i.e., graph-based attention model (GBA) [6] and pointer-generator network (PGN, without coverage mechanism) [4]. We have conducted preliminary experiments on the proposed model with the extractive We also perform human evaluation to evaluate output summaries.…”