Generating Abstractive Summaries with Finetuned Language Models

Gehrmann, Sebastian; Ziegler, Zachary M.; Rushton, Gérard

doi:10.18653/v1/w19-8665

Cited by 18 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent advances in neural text generation have led to significant improvement in the quality of abstractive summarization (Radford et al, 2019;Gehrmann et al, 2019;Lewis et al, 2019). Despite this progress, there are still many limitations facing neural text summarization (Kryscinski et al, 2019), the most serious of which is the tendency to generate summaries that are not factually consistent with the input document; a factually consistent summary only contains statements that can be inferred from the source document.…”

Section: Introductionmentioning

confidence: 99%

Improving Factual Consistency of Abstractive Summarization via Question Answering

Feng¹,

Santos²,

Zhu³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

A commonly observed problem with the stateof-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents. The fact that automatic summarization may produce plausible-sounding yet inaccurate summaries is a major concern that limits its wide application. In this paper we present an approach to address factual consistency in summarization. We first propose an efficient automatic evaluation metric to measure factual consistency; next, we propose a novel learning algorithm that maximizes the proposed metric during model training. Through extensive experiments, we confirm that our method is effective in improving factual consistency and even overall quality of the summaries, as judged by both automatic metrics and human evaluation.

show abstract

Section: Introductionmentioning

confidence: 99%

Improving Factual Consistency of Abstractive Summarization via Question Answering

Feng¹,

Santos²,

Zhu³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…The authors of the Reddit TIFU dataset suggested their own multi-level memory networks [43] that performed better than basic seq2seq and extractive models available by 2019. Within the 2019 TL;DR challenge, authors [44] compared LSTM, LSTM + copy, Transformer, Transformer + copy, and Transformer + pretrained models for n-gram abstractiveness in summarizations. The results have shown that the pretrained Transformer model unnecessarily outperformed the ground truth for news summarizations, but worked best for the Reddit data.…”

Section: Previous Applications Of Deep-learning Summarization Models ...mentioning

confidence: 99%

Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages

2022

View full text Add to dashboard Cite

Abstractive summarization is a technique that allows for extracting condensed meanings from long texts, with a variety of potential practical applications. Nonetheless, today’s abstractive summarization research is limited to testing the models on various types of data, which brings only marginal improvements and does not lead to massive practical employment of the method. In particular, abstractive summarization is not used for social media research, where it would be very useful for opinion and topic mining due to the complications that social media data create for other methods of textual analysis. Of all social media, Reddit is most frequently used for testing new neural models of text summarization on large-scale datasets in English, without further testing on real-world smaller-size data in various languages or from various other platforms. Moreover, for social media, summarizing pools of texts (one-author posts, comment threads, discussion cascades, etc.) may bring crucial results relevant for social studies, which have not yet been tested. However, the existing methods of abstractive summarization are not fine-tuned for social media data and have next-to-never been applied to data from platforms beyond Reddit, nor for comments or non-English user texts. We address these research gaps by fine-tuning the newest Transformer-based neural network models LongFormer and T5 and testing them against BART, and on real-world data from Reddit, with improvements of up to 2%. Then, we apply the best model (fine-tuned T5) to pools of comments from Reddit and assess the similarity of post and comment summarizations. Further, to overcome the 500-token limitation of T5 for analyzing social media pools that are usually bigger, we apply LongFormer Large and T5 Large to pools of tweets from a large-scale discussion on the Charlie Hebdo massacre in three languages and prove that pool summarizations may be used for detecting micro-shifts in agendas of networked discussions. Our results show, however, that additional learning is definitely needed for German and French, as the results for these languages are non-satisfactory, and more fine-tuning is needed even in English for Twitter data. Thus, we show that a ‘one-for-all’ neural-network summarization model is still impossible to reach, while fine-tuning for platform affordances works well. We also show that fine-tuned T5 works best for small-scale social media data, but LongFormer is helpful for larger-scale pool summarizations.

show abstract

“…To analyze the abstraction level of TLDRHQ dataset, we plot the percentage of novel n-grams within the TLDR summary (See et al, 2017) in Figure 5 (b), as well as the TLDR's n-gram abstractiveness (Gehrmann et al, 2019) in Figure 5 (c) over the all instances in TLDRHQ dataset. As indicated, there are quite a large proportion of novel n-gram words appeared in the TLDR summary as the heat extent is mostly concentrated in the upper half of the y-axis.…”

Section: Dataset Analysismentioning

confidence: 99%

TLDR9+: A Large Scale Resource for Extreme Summarization of Social Media Posts

Sotudeh¹,

Deilamsalehy²,

Dernoncourt³

et al. 2021

Proceedings of the Third Workshop on New Frontiers in Summarization

View full text Add to dashboard Cite

Recent models in developing summarization systems consist of millions of parameters and the model performance is highly dependent on the abundance of training data. While most existing summarization corpora contain data in the order of thousands to one million, generation of large-scale summarization datasets in order of couple of millions is yet to be explored. Practically, more data is better at generalizing the training patterns to unseen data. In this paper, we introduce TLDR9+ -a largescale summarization dataset-containing over 9 million training instances extracted from Reddit discussion forum (https://github. com/sajastu/reddit_collector). This dataset is specifically gathered to perform extreme summarization (i.e., generating onesentence summary in high compression and abstraction) and is more than twice larger than the previously proposed dataset. We go one step further and with the help of human annotations, we distill a more finegrained dataset by sampling High-Quality instances from TLDR9+ and call it TLDRHQ dataset. We further pinpoint different state-ofthe-art summarization models on our proposed datasets.

show abstract

Generating Abstractive Summaries with Finetuned Language Models

Cited by 18 publications

References 13 publications

Improving Factual Consistency of Abstractive Summarization via Question Answering

Improving Factual Consistency of Abstractive Summarization via Question Answering

Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages

TLDR9+: A Large Scale Resource for Extreme Summarization of Social Media Posts

Contact Info

Product

Resources

About