Key phrase aware transformer for abstractive summarization

Liu, Shuaiqi; Cao, Jiannong; Yang, Ruosong; Wen, Zhiyuan

doi:10.1016/j.ipm.2022.102913

Cited by 18 publications

(6 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works aim to reduce hallucinations or improve the factual consistency of abstractive summarizers, e.g., employing content planning [21], reinforcement learning [22], or constraining the generation [23]. Abstractive summarizers could also be guided, for instance, to work better on aggregating semantic information [8], with specific topics [24], or to represent better the keywords and relationships among the entities [25,26].…”

Section: Related Workmentioning

confidence: 99%

Abstractive Summarizers Become Emotional on News Summarization

Ahuir,

González,

Hurtado

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Emotions are central to understanding contemporary journalism; however, they are overlooked in automatic news summarization. Actually, summaries are an entry point to the source article that could favor some emotions to captivate the reader. Nevertheless, the emotional content of summarization corpora and the emotional behavior of summarization models are still unexplored. In this work, we explore the usage of established methodologies to study the emotional content of summarization corpora and the emotional behavior of summarization models. Using these methodologies, we study the emotional content of two widely used summarization corpora: Cnn/Dailymail and Xsum, and the capabilities of three state-of-the-art transformer-based abstractive systems for eliciting emotions in the generated summaries: Bart, Pegasus, and T5. The main significant findings are as follows: (i) emotions are persistent in the two summarization corpora, (ii) summarizers approach moderately well the emotions of the reference summaries, and (iii) more than 75% of the emotions introduced by novel words in generated summaries are present in the reference ones. The combined use of these methodologies has allowed us to conduct a satisfactory study of the emotional content in news summarization.

show abstract

Section: Related Workmentioning

confidence: 99%

Abstractive Summarizers Become Emotional on News Summarization

Ahuir,

González,

Hurtado

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…These datasets only focus on the text, regard tabular data as noises, and filter them out. Previous summarization methods can be generally classified into two categories: extractive (Erkan and Radev, 2004;Mihalcea and Tarau, 2004) and abstractive (Nallapati et al, 2016;Zhang et al, 2020;Liu et al, 2022b) summarization methods. To model longer input sequences with limited GPU memory, Huang et al (2021) compare various efficient attention mechanisms for the encoder and propose an encoder-decoder attention named Hepos.…”

Section: Automatic Document Summarizationmentioning

confidence: 99%

Long Text and Multi-Table Summarization: Dataset and Method

Liu¹,

Cao²,

Yang³

et al. 2022

Findings of the Association for Computational Linguistics: EMNLP 2022

View full text Add to dashboard Cite

Automatic document summarization aims to produce a concise summary covering the input document's salient information. Within a report document, the salient information can be scattered in the textual and non-textual content. However, existing document summarization datasets and methods usually focus on the text and filter out the non-textual content. Missing tabular data can limit produced summaries' informativeness, especially when summaries require covering quantitative descriptions of critical metrics in tables. Existing datasets and methods cannot meet the requirements of summarizing long text and multiple tables in each report. To deal with the scarcity of available data, we propose FINDSum, the first largescale dataset for long text and multi-table summarization. Built on 21,125 annual reports from 3,794 companies, it has two subsets for summarizing each company's results of operations and liquidity. To summarize the long text and dozens of tables in each report, we present three types of summarization methods. Besides, we propose a set of evaluation metrics to assess the usage of numerical information in produced summaries. Dataset analyses and experimental results indicate the importance of jointly considering input textual and tabular data when summarizing report documents.

show abstract

“…As for the second question we should say that most scholars have considered these tasks independent. However, a few have argued that they are interrelated and can benefit each other [3], [4], [23]- [27]. There are a number of reasons behind this argument.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, interacting with RQE can be fruitful since it ensures the generated summaries are logically entailed by their source texts [23], [28]- [30]. Another reason is that generating concise summaries requires a greater focus on the main topics of questions, which existing QS models often fail to do [27]. Consequently, while duplicate information are repeatedly given attention, essential phrases may go unnoticed [26], [27].…”

Section: Introductionmentioning

confidence: 99%

“…Another reason is that generating concise summaries requires a greater focus on the main topics of questions, which existing QS models often fail to do [27]. Consequently, while duplicate information are repeatedly given attention, essential phrases may go unnoticed [26], [27]. Therefore, as tags are good indicators for the main topics of questions, joint learning of TG and QS would provide the model with essential information in encoding layers [31].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ReQuEST: A Small-Scale Multi-Task Model for Community Question-Answering Systems

Aftabi,

Seyyedi,

Maleki

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The burgeoning popularity of community question-answering platforms as an informationseeking strategy has prompted researchers to look for ways to save response time and effort, among which question entailment recognizing, question summarizing, and question tagging are prominent. However, none has investigated the implicit relations between these tasks and the benefits their interaction could provide. In this study, ReQuEST, a novel multi-task model based on bidirectional auto-regressive transformers (BART), is introduced to recognize question entailment, summarize questions respecting given queries, and tag questions with primary topics, simultaneously. ReQuEST comprises one shared encoder representing input sequences, two half-shared decoders providing intermediate presentations, and three task-specific heads producing summaries, tags, and entailed questions. A lightweight fine-tuning technique and a weighted loss function help us learn model parameters efficiently. With roughly 187k learning parameters, ReQuEST is almost half the size of BARTlarge and is two-thirds smaller than its multi-task counterparts. Empirical experiments on standard summarization datasets reveal that ReQuEST outperforms competitors on Debatepedia with a Rouge-L of 46.77 and has persuasive performance with a Rouge-L of 37.37 on MeQSum. On MediQA-RQE as a medical benchmark for entailment recognition, ReQuEST is also comparable in accuracy with state-of-the-art systems without being pre-trained on domain-specific datasets.

show abstract

Key phrase aware transformer for abstractive summarization

Cited by 18 publications

References 45 publications

Abstractive Summarizers Become Emotional on News Summarization

Abstractive Summarizers Become Emotional on News Summarization

Long Text and Multi-Table Summarization: Dataset and Method

ReQuEST: A Small-Scale Multi-Task Model for Community Question-Answering Systems

Contact Info

Product

Resources

About