A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion

Lebanoff, Logan; Dernoncourt, Franck; Kim, Doo Soon; Chang, Walter; Liu, Fei

doi:10.48550/arxiv.2010.03722

Cited by 4 publications

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

FFCI: A Framework for Interpretable Automatic Evaluation of Summarization

Koto

Lau²,

Baldwin³

2022

jair

View full text Add to dashboard Cite

In this paper, we propose FFCI, a framework for fine-grained summarization evaluation that comprises four elements: faithfulness (degree of factual consistency with the source), focus (precision of summary content relative to the reference), coverage (recall of summary content relative to the reference), and inter-sentential coherence (document fluency between adjacent sentences). We construct a novel dataset for focus, coverage, and inter-sentential coherence, and develop automatic methods for evaluating each of the four dimensions of FFCI based on cross-comparison of evaluation metrics and model-based evaluation methods, including question answering (QA) approaches, semantic textual similarity (STS), next-sentence prediction (NSP), and scores derived from 19 pre-trained language models. We then apply the developed metrics in evaluating a broad range of summarization models across two datasets, with some surprising findings.

show abstract

FFCI: A Framework for Interpretable Automatic Evaluation of Summarization

Koto

Lau²,

Baldwin³

2022

jair

View full text Add to dashboard Cite

show abstract

Long-Span Summarization via Local Attention and Content Selection

Manakul¹,

Gales²

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Transformer-based models have achieved state-of-the-art results in a wide range of natural language processing (NLP) tasks including document summarization. Typically these systems are trained by fine-tuning a large pretrained model to the target task. One issue with these transformer-based models is that they do not scale well in terms of memory and compute requirements as the input length grows. Thus, for long document summarization, it can be challenging to train or fine-tune these models. In this work, we exploit large pre-trained transformer-based models and address long-span dependencies in abstractive summarization using two methods: local self-attention; and explicit content selection. These approaches are compared on a range of network configurations. Experiments are carried out on standard long-span summarization tasks, including Spotify Podcast, arXiv, and PubMed datasets. We demonstrate that by combining these methods, we can achieve state-of-the-art results on all three tasks in the ROUGE scores. Moreover, without a large-scale GPU card, our approach can achieve comparable or better results than existing approaches. 1

show abstract

Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline

Ernst¹,

Shapira²,

Pasunuru³

et al. 2021

Proceedings of the 25th Conference on Computational Natural Language Learning

View full text Add to dashboard Cite

Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task, notably for generating training data for salience detection. Despite its assessed utility, the alignment step was mostly approached with heuristic unsupervised methods, typically ROUGE-based, and was never independently optimized or evaluated. In this paper, we propose establishing summary-source alignment as an explicit task, while introducing two major novelties: (1) applying it at the more accurate proposition span level, and (2) approaching it as a supervised classification task. To that end, we created a novel training dataset for proposition-level alignment, derived automatically from available summarization evaluation data. In addition, we crowdsourced dev and test datasets, enabling model development and proper evaluation. Utilizing these data, we present a supervised proposition alignment baseline model, showing improved alignmentquality over the unsupervised approach.

show abstract

A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion

Cited by 4 publications

References 28 publications

FFCI: A Framework for Interpretable Automatic Evaluation of Summarization

FFCI: A Framework for Interpretable Automatic Evaluation of Summarization

Long-Span Summarization via Local Attention and Content Selection

Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline

Contact Info

Product

Resources

About