Automatic Annotation of Corpora for Text Summarisation: A Comparative Study

Abstract

Abstract. This paper presents two methods which automatically produce annotated corpora for text summarisation on the basis of human produced abstracts. Both methods identify a set of sentences from the document which conveys the information in the human produced abstract best. The first method relies on a greedy algorithm, whilst the second one uses a genetic algorithm. The methods allow to specify the number of sentences to be annotated, which constitutes an advantage over the existing methods. Comparison between the two approaches investigated here revealed that the genetic algorithm is appropriate in cases where the number of sentences to be annotated is less than the number of sentences in an ideal gold standard with no length restrictions, whereas the greedy algorithm should be used in other cases.

Automatic summarisation: 25 years On

“…Hirschman and Mani (2003) point out that for intrinsic evaluation, measures such as quality and informativeness are recorded, whilst in extrinsic evaluation, post-edit measures, relevance assessment and reading comprehension tests are commonly used. A more detailed classification of types of evaluation methods used in automatic summarisation is discussed in Tucker (1999); Orăsan (2006), but that classification is too specific for the purpose of this article. Although widely used, the appropriateness of intrinsic evaluation was questioned by Spärck-Jones (2001) because there is ‘nothing like a natural summary for a text’.…”