Text Summarization Challenge 2 text summarization evaluation at NTCIR workshop 3

Okumura, Manabu; Fukusima, Takahiro; Nanba, Hidetsugu; Hirao, Tsutomu

doi:10.1145/986278.986284

Cited by 15 publications

(7 citation statements)

References 10 publications

(1 reference statement)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, regarding the intrinsic evaluation task (question-answering task), the goal was to measure the content of a summary with respect to which degree it contained answers to several topic-related questions. The tasks involved in the TSC workshops within NTCIR conferences (Fukushima and Okumura 2001;Okumura et al 2004;Hirao et al 2005) also dealt with the evaluation using intrinsic and extrinsic methods. In the first TSC, three tasks were proposed, consisting of producing summaries given a specific length.…”

Section: Relevant Conferences and Workhopsmentioning

confidence: 99%

Text summarisation in progress: a literature review

2011

View full text Add to dashboard Cite

This paper contains a large literature review in the research field of Text Summarisation (TS) based on Human Language Technologies (HLT). TS helps users manage the vast amount of information available, by condensing documents' content and extracting the most relevant facts or topics included in them. The rapid development of emerging technologies poses new challenges to this research field, which still need to be solved. Therefore, it is essential to analyse its progress over the years, and provide an overview of the past, present and future directions, highlighting the main advances achieved and outlining remaining limitations. With this purpose, several important aspects are addressed within the scope of this survey. On the one hand, the paper aims at giving a general perspective on the state-ofthe-art, describing the main concepts, as well as different summarisation approaches, and relevant international forums. Furthermore, it is important to stress upon the fact that the birth of new requirements and scenarios has led to new types of summaries with specific purposes (e.g. sentiment-based summaries), and novel domains within which TS has proven to be also suitable for (e.g. blogs). In addition, TS is successfully combined with a number of intelligent systems based on HLT (e.g. information retrieval, question answering, and text classification). On the other hand, a deep study of the evaluation of summaries is also conducted in this paper, where the existing methodologies and systems are explained, as well as new research that has emerged concerning the automatic evaluation of summaries' quality. Finally, some thoughts about TS in general and its future will encourage the reader to think of novel approaches, applications and lines to conduct research in the next years. The analysis of these issues allows the reader to have a wide and useful background on the main important aspects of this research field.

show abstract

Section: Relevant Conferences and Workhopsmentioning

confidence: 99%

Text summarisation in progress: a literature review

2011

View full text Add to dashboard Cite

show abstract

“…Activities to promote summarization in Japanese have been undertaken in the Text Summarization Challenges [46], a series of evaluations of text summarization systems with tasks such as single document summarization and topic focused multidocument summarization. Summarization in a cross-lingual environment was studied in the 2001 Johns Hopkins research workshop [65] where evaluation resources and summarization algorithms for English and Chinese were developed [54].…”

Section: Multilingual Summarizationmentioning

confidence: 99%

Automatic Text Summarization: Past, Present and Future

Saggion

Poibeau

2012

Theory and Applications of Natural Language Processing

124

View full text Add to dashboard Cite

Automatic text summarization, the computer-based production of condensed versions of documents, is an important technology for the information society. Without summaries it would be practically impossible for human beings to get access to the ever growing mass of information available online. Although research in text summarization is over fifty years old, some efforts are still needed given the insufficient quality of automatic summaries and the number of interesting summarization topics being proposed in different contexts by end users ("domain-specific summaries", "opinion-oriented summaries", "update summaries", etc.). This paper gives a short overview of summarization methods and evaluation.

show abstract

“…The other 12 events were used as the training corpus. We adopted, the subjective evaluation method used in NTCIR-TSC2 [Okumura et al 2004]. Human judges evaluated and ranked all n summaries on a 1 to n scale, where 1 was best, 2 was second, and n was worst.…”

Section: Data Set and Evaluation Metricsmentioning

confidence: 99%

Multidocument Summary Generation

Kuo

Chen

2008

ACM Transactions on Asian Language Information Processing

View full text Add to dashboard Cite

generation for multiple documents poses a number of issues including sentence selection, sentence ordering, and sentence reduction over single-document summarization. In addition, the temporal resolution among extracted sentences is also important. This article considers informative words and event words to deal with multidocument summarization. These words indicate the important concepts and relationships in a document or among a set of documents, and can be used to select salient sentences. We present a temporal resolution algorithm, using focusing time and coreference chains, to convert Chinese temporal expressions in a document into calendrical forms. Moreover, we consider the last calendrical form of a sentence as a sentence time stamp to address sentence ordering. Informative words, event words, and temporal words are introduced to a sentence reduction algorithm, which deals with both length constraints and information coverage. Experiments on Chinese-news data sets show significant improvements of both information coverage and readability.

show abstract

Text Summarization Challenge 2 text summarization evaluation at NTCIR workshop 3

Cited by 15 publications

References 10 publications

Text summarisation in progress: a literature review

Text summarisation in progress: a literature review

Automatic Text Summarization: Past, Present and Future

Multidocument Summary Generation

Contact Info

Product

Resources

About