Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Abstract

Automatic multi-document summarization needs to find representative sentences not only b y sentence distrib ution to select the most important sentence b ut also b y how informative a term is in a sentence. Sentence distrib ution is suitab le for ob taining important sentences b y determining frequent and well-spread words in the corpus b ut ignores the grammatical information that indicates instructive content. The presence or ab sence of informative content in a sentence can b e indicated b y grammatical information which is carried b y part of speech (POS) lab els. In this paper, we propose a new sentence weighting method b y incorporating sentence distrib ution and POS tagging for multi -document summarization. Similarity-based Histogram Clustering (SHC) is used to cluster sentences in the data set. Cluster ordering is b ased on cluster importance to determine the important clusters. Sentence extraction b ased on sentence distrib ution and POS tagging is introduced to extract the representative sentences from the ordered clusters. The results of the experiment on the Document Unde rstanding Conferences (DUC) 2004 are compared with those of the Sentence Distrib ution Method. Our proposed method achieved b etter results with an increasing rate of 5.41% on ROUGE-1 and 0.62% on ROUGE-2.

Exploring the Performance of Tagging for the Classical and the Modern Standard Arabic

“…(v) Other PoS tagging based applications include: semantic role labeling [10], speech synthesis [11], speech recognition [12], information extraction [13], summarization [14], sentiment analysis also called opinion mining [15], diacritization [16], software engineering [17], question answering [18], translation [19], plagiarism detection [20], key phrases extraction [21], ontology [22], and extracting Arabic noun compound [23].…”

Peringkasan Teks Multi-Dokumen Berdasarkan Metode Sentence Extraction Dan Word Sense Disambiguation

“…Local sentence distribution digunakan untuk menentukan penting atau tidaknya suatu kalimat dalam klaster. Kalimat yang terdiri dari kata-kata yang banyak tersebar dalam suatu klaster akan dianggap lebih penting dan mendapat posisi yang lebih tinggi dalam suatu klaster [12]. Sedangkan global sentence distribution digunakan untuk menentukan penting atau tidaknya suatu kalimat dalam kumpulan klaster.…”

“…Sedangkan global sentence distribution digunakan untuk menentukan penting atau tidaknya suatu kalimat dalam kumpulan klaster. Suatu kalimat yang memiliki sebaran term yang tinggi dalam klaster namun memiliki sebaran yang rendah pada klaster lainnya dianggap lebih penting dan memiliki nilai global sentence distribution yang lebih tinggi [12].…”