Detecting (Un)Important Content for Single-Document News Summarization

Yang, Yinfei; Bao, Forrest Sheng; Nenkova, Ani

doi:10.18653/v1/e17-2112

Cited by 13 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Absolute vs Relative Summary Ranking. In relative assessment of summarization, annotators are shown two or more summaries and are asked to rank them according to the dimension at question (Yang et al, 2017;Chen and Bansal, 2018;Narayan et al, 2018a;Guo et al, 2018;Krishna and Srinivasan, 2018). The relative assessment is often done using the paired comparison (Thurstone, 1994) or the best-worst scaling (Woodworth and G, 1991;Louviere et al, 2015), to improve inter-annotator agreement.…”

Section: Literature Reviewmentioning

confidence: 99%

HighRES: Highlight-based Reference-less Evaluation of Summarization

Hardy¹,

Narayan²,

Vlachos³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

There has been substantial progress in summarization research enabled by the availability of novel, often large-scale, datasets and recent advances on neural network-based approaches. However, manual evaluation of the system generated summaries is inconsistent due to the difficulty the task poses to human non-expert readers. To address this issue, we propose a novel approach for manual evaluation, HIGHlight-based Reference-less Evaluation of Summarization (HIGHRES), in which summaries are assessed by multiple annotators against the source document via manually highlighted salient content in the latter. Thus summary assessment on the source document by human judges is facilitated, while the highlights can be used for evaluating multiple systems. To validate our approach we employ crowd-workers to augment with highlights a recently proposed dataset and compare two state-of-the-art systems. We demonstrate that HIGHRES improves inter-annotator agreement in comparison to using the source document directly, while they help emphasize differences among systems that would be ignored under other evaluation approaches. 1

show abstract

Section: Literature Reviewmentioning

confidence: 99%

HighRES: Highlight-based Reference-less Evaluation of Summarization

Hardy¹,

Narayan²,

Vlachos³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…As we will shortly see, the classifier is impressively accurate on instances in which the annotators agreed in their initial annotation and quite poor on the leads that required adjudication. These findings suggest that in future work in may be beneficial to develop a classifier for sentence-level prediction (Yang, Bao, & Nenkova, 2017) of content-density, which would be helpful for characterizing leads that mix informative and entertaining sentences. Another clear alternative is to develop a classifier to predict that a text is ambiguous in terms of its content-density status.…”

Section: Basic Setmentioning

confidence: 95%

Combining Lexical and Syntactic Features for Detecting Content-Dense Texts in News

Yang¹,

Nenkova²

2017

jair

Self Cite

View full text Add to dashboard Cite

Content-dense news report important factual information about an event in direct, succinct manner. Information seeking applications such as information extraction, question answering and summarization normally assume all text they deal with is content-dense. Here we empirically test this assumption on news articles from the business, U.S. international relations, sports and science journalism domains. Our findings clearly indicate that about half of the news texts in our study are in fact not content-dense and motivate the development of a supervised content-density detector. We heuristically label a large training corpus for the task and train a two-layer classifying model based on lexical and unlexicalized syntactic features. On manually annotated data, we compare the performance of domain-specific classifiers, trained on data only from a given news domain and a general classifier in which data from all four domains is pooled together. Our annotation and prediction experiments demonstrate that the concept of content density varies depending on the domain and that naive annotators provide judgement biased toward the stereotypical domain label. Domain-specific classifiers are more accurate for domains in which contentdense texts are typically fewer. Domain independent classifiers reproduce better naive crowdsourced judgements. Classification prediction is high across all conditions, around 80%.

show abstract

“…Early work on extractive summarisation centers solely around simple to compute statistics, for instance word frequency [3] (Luhn, 1958), location in document [4] ABSTRACT (Baxendale, 1958), and TF-IDF [5] (Salton et al, 1996). Exploration of more aspects such as sentence position [6](Yang et al, 2017), sentence length [7](Radev et al, 2004), words in the title, presence of formal nouns, places or things, word recurrence [8] (Nenkova et al, 2006) a graph-based ranking model for text processing [11]( Rada Mihalcea and Paul Tarau). A stochastic graph-based method was proposed by Dragomir R. Radev [12]for the computation of relative importance of textual portions in NLP.…”

Section: Iirelated Workmentioning

confidence: 99%

Implementing Supervised Approach to Summarization of Research Papers

Awasth¹

2020

IJMTST

View full text Add to dashboard Cite

Using automatic text summarization we can reduce a document to its main information or to what is known as crux of the document .Recent research in this zone has zeroed in on neural ways to deal with summarisation, which can be very data hungry. This paper aims to explore a quicker way by implementing a supervised-learning based extractive summarisation system for the summarisation of research papers. This paper also explores the possibility of any section, in a research paper being the prime section to generate summaries by utilizing ROUGE scores. An easy to implement and intuitive model is developed using glove embeddings and doc2vec to encode sentences and documents in their local and global context producing grammatically coherent summaries.

show abstract

Detecting (Un)Important Content for Single-Document News Summarization

Cited by 13 publications

References 11 publications

HighRES: Highlight-based Reference-less Evaluation of Summarization

HighRES: Highlight-based Reference-less Evaluation of Summarization

Combining Lexical and Syntactic Features for Detecting Content-Dense Texts in News

Implementing Supervised Approach to Summarization of Research Papers

Contact Info

Product

Resources

About