Document Summarization Based on Data Reconstruction

Abstract

Document summarization is of great value to many real world applications, such as snippets generation for search results and news headlines generation. Traditionally, document summarization is implemented by extracting sentences that cover the main topics of a document with a minimum redundancy. In this paper, we take a different perspective from data reconstruction and propose a novel framework named Document Summarization based on Data Reconstruction (DSDR). Specifically, our approach generates a summary which consist of those sentences that can best reconstruct the original document. To model the relationship among sentences, we introduce two objective functions: (1) linear reconstruction, which approximates the document by linear combinations of the selected sentences; (2) nonnegative linear reconstruction, which allows only additive, not subtractive, linear combinations. In this framework, the reconstruction error becomes a natural criterion for measuring the quality of the summary. For each objective function, we develop an efficient algorithm to solve the corresponding optimization problem. Extensive experiments on summarization benchmark data sets DUC 2006 and DUC 2007 demonstrate the effectiveness of our proposed approach.

Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization

“…Summarization systems can be generally categorized into two paradigms: extractive and abstractive. Extractive systems extract certain sentences and clauses from input, for example, based on salient features (Zhou and Rush, 2019) or feature construction (He et al, 2012). Abstraction systems generate new utterances as the summary, e.g., by sequence-to-sequence models trained in a supervised way (Zhang et al, 2020;Liu et al, 2021b).…”

Unsupervised Summarization for Chat Logs with Topic-Oriented Ranking and Context-Aware Auto-Encoders

“…A couple of works proposed extractive methods for unsupervised summarization, which generally assign salient scores to sentences in a document and select the top-ranked ones to form the summary. Typical methods are based on word frequency (Nenkova and Vanderwende 2005), topic modeling (Harabagiu and Lacatusu 2005), cluster centroid (Radev et al 2004;Rossiello et al 2017), sentence graph (Erkan and Radev 2004;Zheng and Lapata 2019), Integer Linear Programming (ILP) optimization (McDonald 2007;Gillick et al 2009), and sparse coding (He et al 2012;Liu et al 2015). Recently, abstractive approaches have been proposed due to the success of deep neural models, where the autoencoder framework has been applied (Miao and Blunsom 2016;Fevry and Phang 2018;Chu and Liu 2019;Liu et al 2019b).…”

Submodular Span, with Applications to Conditional Data Summarization

“…The methods based on data reconstruction, for example DSDR (He et al 2012) reconstructs each sentence by a non-negative linear combination of summary sentences and then uses sparse coding to select summary sentences that minimize the document reconstruction error. SpOpt (Yao, Wan, and Xiao 2015) adds a sentence dissimilarity term to the objective to maximize diversity.…”

“…In machine learning, and fields such as natural language processing (NLP) and information retrieval (IR), various approaches have been used to solve this problem. Query-based MDS can be in either supervised where labels are available and a training phase occurs, for example (Lin andBilmes 2011, 2012) or unsupervised where there are no target labels to train on as in (He et al 2012;Yao, Wan, and Xiao 2015;Feigenblat et al 2017). In query-based extractive video summarization, recent methods include snippet selection using sequential and hierarchical Determinantal Point Processes (DPP) (Sharghi, Gong, and Shah 2016;Sharghi, Laurel, and Gong 2017).…”