Multi-document Automatic Text Summarization Using Entropy Estimates

Abstract

Abstract. This paper describes a sentence ranking technique using entropy measures, in a multi-document unstructured text summarization application. The method is topic specific and makes use of a simple language independent training framework to calculate entropies of symbol units. The document set is summarized by assigning entropy-based scores to a reduced set of sentences obtained using a graph representation for sentence similarity. The performance is seen to be better than some of the common statistical techniques, when applied on the same data set. Commonly used measures like precision, recall and f-score have been modified and used as a new set of measures for comparing the performance of summarizers. The rationale behind such a modification is also presented. Experimental results are presented to illustrate the relevance of this method in cases where it is difficult to have language specific dictionaries, translators and document-summary pairs for training.

Information-Balance-Aware Approximated Summarization of Data Provenance

“…On the other hand, although the concept of entropy and its related derivatives have been successfully applied to a wide range of problems related to summary and compression [19][20][21][22][23], our work is the first attempt to try to involve the computation and control of entropy measurements with the user requirement specification during approximated provenance summarization. We believe that by allowing more flexible control of approximated data provenance summarization using entropy measurements, a wide scope of provenancerelated tasks, for example, provenance based access control rules retrieval [24], provenance visualization [25], and provenance storage [26,27], can be performed both more effectively and more efficiently.…”

Multi-document Summarization Using Minimum Distortion

“…The previous information-theoretic methods for document summarization are based on information distance [17] and entropy estimates [24]. [24] ranks sentences by calculating entropies of symbol units.…”

“…The previous information-theoretic methods for document summarization are based on information distance [17] and entropy estimates [24]. [24] ranks sentences by calculating entropies of symbol units. [17] uses the theory of Kolmogorov complexity and generates summaries that have the minimum information distance with the original documents.…”

EnDSUM: Entropy and Diversity based Disaster Tweet Summarization

“…Therefore, EnDSUM can generate the summary automatically without explicitly identifying the category of a tweet. Although there are few single and multiple document summarization approaches [1,11,17,19,21] that have highlighted the relevance of entropy based selection of sentences into summary, those approaches are not directly applicable to disaster tweets. The reason being the informal structure of tweets, absence of storyline in tweets and the high vocabulary diversity in user generated tweets.…”