Diversity driven Attention Model for Query-based Abstractive Summarization

Nema, Preksha; Khapra, Mitesh M.; Laha, Anirban; Ravindran, Balaraman

doi:10.48550/arxiv.1704.08300

Cited by 16 publications

(34 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Datasets We use multiple QA datasets, including SQuAD (Rajpurkar et al, 2016), NewsQA (Trischler et al, 2016), TriviaQA (Joshi et al, 2017), SearchQA (Dunn et al, 2017), HotpotQA (Yang et al, 2018) and NaturalQuestions (Kwiatkowski et al, 2019) to train HLTC-MRQA, following Su et al (2019). We evaluate our model on the Debatepedia dataset (Nema et al, 2017) and DUC2005-7 dataset (in Appendix).…”

Section: Methodsmentioning

confidence: 99%

“…QFS is a more complex task that aims to generate a summary according to the query and its relevant document(s). Nema et al (2017) proposed an encode-attend-decode system with an additional query attention mechanism and diversity-based attention mechanism to generate a more queryrelevant summary. Baumel et al (2018) rated query relevance into a pre-trained abstractive summarizer to make the model aware of the query, while Xu and Lapata (2020a) discovered a new type of connection between generic summaries and QFS queries, and provided a universal representation for them which allows generic summarization data to be further exploited for QFS.…”

Section: Related Workmentioning

confidence: 99%

“…A Adapting QFS-BART to DUC 2005-7 DUC 2005-7 are datasets for the multi-document query focused summarization (QFS) task. As shown in the Table 4, the documents and summaries of the DUC datasets are extremely longer than those in the Debatepedia (Nema et al, 2017) dataset. We thus need to adapt the QFS-BART model to handle the multi-document scenario and produce longer output.…”

Section: A1 Answer Retrievingmentioning

confidence: 99%

“…Other work on abstractive QFS incorporated the query relevance into existing neural summarization models (Nema et al, 2017;Baumel et al, 2018). The closest work to ours was done by (Su et al, 2020) and (Xu and Lapata, 2020a,b), who leveraged an external question answering (QA) module in a pipeline framework to take into consideration the answer relevance of the generated summary.…”

Section: Introductionmentioning

confidence: 99%

“…We leverage a state-of-the-art QA model (Su et al, 2019) to predict the answer relevance of the given source documents to the query, then further incorporate the answer relevance into the BARTbased generation model. We conduct empirical experiments on the Debatepedia dataset, one of the first large-scale QFS datasets (Nema et al, 2017), and achieve the new state-of-the-art performance on the ROUGE metrics compared to all previously published work.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance

Su¹,

Yu²,

Fung³

2021

Preprint

View full text Add to dashboard Cite

Query focused summarization (QFS) models aim to generate summaries from source documents that can answer the given query. Most previous work on QFS only considers the query relevance criterion when producing the summary. However, studying the effect of answer relevance in the summary generating process is also important. In this paper, we propose QFS-BART, a model that incorporates the explicit answer relevance of the source documents given the query via a question answering model, to generate coherent and answerrelated summaries. Furthermore, our model can take advantage of large pre-trained models which improve the summarization performance significantly. Empirical results on the Debatepedia dataset show that the proposed model achieves the new state-of-the-art performance. 1 * * The two authors contribute equally. 1 The code is released at: https://github.com/ HLTCHKUST/QFS On the other hand, recent neural summarization models (Paulus et al., 2017;Gehrmann et al., 2018; have achieved remarkable performance in generic abstractive summarization by taking advantage of large pre-trained language

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: A1 Answer Retrievingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance

Su¹,

Yu²,

Fung³

2021

Preprint

View full text Add to dashboard Cite

show abstract

Abstractive Summarization Improved by WordNet-Based Extractive Sentences

Xie

Ren

et al. 2018

Natural Language Processing and Chinese Computing

View full text Add to dashboard Cite

Recently, the seq2seq abstractive summarization models have achieved good results on the CNN/Daily Mail dataset. Still, how to improve abstractive methods with extractive methods is a good research direction, since extractive methods have their potentials of exploiting various efficient features for extracting important sentences in one text. In this paper, in order to improve the semantic relevance of abstractive summaries, we adopt the WordNet based sentence ranking algorithm to extract the sentences which are most semantically to one text. Then, we design a dual attentional seq2seq framework to generate summaries with consideration of the extracted information. At the same time, we combine pointer-generator and coverage mechanisms to solve the problems of out-of-vocabulary (OOV) words and duplicate words which exist in the abstractive models. Experiments on the CNN/Daily Mail dataset show that our models achieve competitive performance with the state-of-theart ROUGE scores. Human evaluations also show that the summaries generated by our models have high semantic relevance to the original text.

show abstract

Conditionally Learn to Pay Attention for Sequential Visual Task

He¹,

Cao²,

Zhang³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Sequential visual task usually requires to pay attention to its current interested object conditional on its previous observations. Different from popular soft attention mechanism, we propose a new attention framework by introducing a novel conditional global feature which represents the weak feature descriptor of the current focused object. Specifically, for a standard CNN (Convolutional Neural Network) pipeline, the convolutional layers with different receptive fields are used to produce the attention maps by measuring how the convolutional features align to the conditional global feature. The conditional global feature can be generated by different recurrent structure according to different visual tasks, such as a simple recurrent neural network for multiple objects recognition, or a moderate complex language model for image caption. Experiments show that our proposed conditional attention model achieves the best performance on the SVHN (Street View House Numbers) dataset with / without extra bounding box; and for image caption, our attention model generates better scores than the popular soft attention model. Recent successes in machine translation[1], speech recognition [2], and image caption [3] have witnessed the important role of attention mechanism. In computer vision, like human visual system, attention does not need to focus on the whole image, but only on the salient areas of the image. For example, [4], [5], [6], [7] embedded attention mechanism into image caption which enables the model to learn to automatically generate a caption describing the content of an image. Subsequently, attention approaches were introduced into the emerging visual question answering task (VQA) which greatly improved the overall performance [8] [9] [7]. Recently, [10] proposed a novel end-to-end trainable attention module for convolutional neural network architectures. The core idea of their work lies in estimating the attention maps by measuring how the local convolutional feature aligns to the global feature, which is different from the 1 arXiv:1911.04365v1 [cs.CV]

show abstract

Diversity driven Attention Model for Query-based Abstractive Summarization

Cited by 16 publications

References 0 publications

Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance

Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance

Abstractive Summarization Improved by WordNet-Based Extractive Sentences

Conditionally Learn to Pay Attention for Sequential Visual Task

Contact Info

Product

Resources

About