DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization

Mao, Ziming; Wu, Chen; Ni, Ansong; Zhang, Yusen; Zhang, Rui; Yu, Tao; Deb, Budhaditya; Zhu, Chenguang; Awadallah, Ahmed Hassan; Radev, Dragomir

doi:10.18653/v1/2022.acl-long.118

Cited by 18 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Its training-stage content selection relies on ROUGE. DYLE + RoBERTa + BART (Mao et al . 2022), a dynamic latent extraction approach for abstractive LDS. DYLE is made up of an extractor which is initialized with RoBERTa (Liu et al ., 2019), and a generator which is initialized with BART. LED + BART (Beltagy et al ., 2020), fine-tuning BART for a Longformer variant.…”

Section: Methodsmentioning

confidence: 99%

Joint learning of text alignment and abstractive summarization for long documents via unbalanced optimal transport

Shen

Lam

et al. 2023

Nat. Lang. Eng.

View full text Add to dashboard Cite

Recently, neural abstractive text summarization (NATS) models based on sequence-to-sequence architecture have drawn a lot of attention. Real-world texts that need to be summarized range from short news with dozens of words to long reports with thousands of words. However, most existing NATS models are not good at summarizing long documents, due to the inherent limitations of their underlying neural architectures. In this paper, we focus on the task of long document summarization (LDS). Based on the inherent section structures of source documents, we divide an abstractive LDS problem into several smaller-sized problems. In this circumstance, how to provide a less-biased target summary as the supervision for each section is vital for the model’s performance. As a preliminary, we formally describe the section-to-summary-sentence (S2SS) alignment for LDS. Based on this, we propose a novel NATS framework for the LDS task. Our framework is built based on the theory of unbalanced optimal transport (UOT), and it is named as UOTSumm. It jointly learns three targets in a unified training objective, including the optimal S2SS alignment, a section-level NATS summarizer, and the number of aligned summary sentences for each section. In this way, UOTSumm directly learns the text alignment from summarization data, without resorting to any biased tool such as ROUGE. UOTSumm can be easily adapted to most existing NATS models. And we implement two versions of UOTSumm, with and without the pretrain-finetune technique. We evaluate UOTSumm on three publicly available LDS benchmarks: PubMed, arXiv, and GovReport. UOTSumm obviously outperforms its counterparts that use ROUGE for the text alignment. When combined with UOTSumm, the performance of two vanilla NATS models improves by a large margin. Besides, UOTSumm achieves better or comparable performance when compared with some recent strong baselines.

show abstract

Section: Methodsmentioning

confidence: 99%

Joint learning of text alignment and abstractive summarization for long documents via unbalanced optimal transport

Shen

Lam

et al. 2023

Nat. Lang. Eng.

View full text Add to dashboard Cite

show abstract

“…TextRank [11] 16.27 2.69 15.41 PGNet [12] 28.74 5.98 25.13 BART [13] 29.20 6.37 25.49 HMNet [14] 32.29 8.67 28.17 SUMM N [15] 34.03 9.28 29.48 DYLE [17] 34.42 9.71 30.10 DialogLM [18] 33…”

Section: Models Rouge-1 Rouge-2 Rouge-lmentioning

confidence: 99%

Query-Utterance Attention with Joint modeling for Query-Focused Meeting Summarization

Liu¹,

Duan²,

Xiao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Query-focused meeting summarization (QFMS) aims to generate summaries from meeting transcripts in response to a given query. Previous works typically concatenate the query with meeting transcripts and implicitly model the query relevance only at the token level with attention mechanism. However, due to the dilution of key query-relevant information caused by long meeting transcripts, the original transformerbased model is insufficient to highlight the key parts related to the query. In this paper, we propose a query-aware framework with joint modeling token and utterance based on Query-Utterance Attention. It calculates the utterance-level relevance to the query with a dense retrieval module. Then both token-level query relevance and utterance-level query relevance are combined and incorporated into the generation process with attention mechanism explicitly. We show that the query relevance of different granularities contributes to generating a summary more related to the query. Experimental results on the QMSum dataset show that the proposed model achieves new state-of-the-art performance.

show abstract

“…LoBART [13] uses local attention and content selection strategy to address long-span dependency problem. DYLE [14] jointly trains an extractor and a generator to enable snippet-level attention during decoding. SSN-DM [19] employs a sliding window to extract summary sentences segment by segment.…”

Section: Summarizationmentioning

confidence: 99%

“…There is an increasing demand for quickly grasping the main content of scientific articles because of the rapid advancement of modern technology. Hence, scientific summarization [13,14,19] is proposed to transform a long scientific article into a concise summary. Considering that it's challenging to include all the important information in a single, typical summary, scientific faceted summarization task [1] is introduced to provide multiple summaries from different facets.…”

Section: Introductionmentioning

confidence: 99%

CPLR-SFS: Contrastive Prompt Learning to Reduce Redundancy for Scientific Faceted Summarization

Xiong

Mao

Yang

et al. 2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Scientific faceted summarization is a task to generate four summaries for a scientific article from different facets, including Purpose, Method, Findings, and Value. Existing works usually generate summary independently for each facet using pre-training or prompt-training paradigms. However, these works tend to produce duplicate content among different facets of the same scientific article, because they do not consider the relation among the aforementioned four facets. To solve the redundancy problem, we propose a novel Contrastive Prompt Learning method to Reduce redundancy for Scientific Faceted Summarization, named CPLR-SFS, to generate concise and less-overlapping faceted summaries. Specifically, CPLR-SFS receives the facet-specific prompt to guide the generation and utilizes the faceted contrastive loss for better distinguishing different faceted summaries in semantic space. Extensive experiments on the FacetSum dataset demonstrate that the proposed model can generate better faceted summaries than the state-of-the-art baselines with less redundancy.

show abstract

DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization

Cited by 18 publications

References 21 publications

Joint learning of text alignment and abstractive summarization for long documents via unbalanced optimal transport

Joint learning of text alignment and abstractive summarization for long documents via unbalanced optimal transport

Query-Utterance Attention with Joint modeling for Query-Focused Meeting Summarization

CPLR-SFS: Contrastive Prompt Learning to Reduce Redundancy for Scientific Faceted Summarization

Contact Info

Product

Resources

About