Graph Ranking on Maximal Frequent Sequences for Single Extractive Text Summarization

Abstract

Abstract. We suggest a new method for the task of extractive text summarization using graph-based ranking algorithms. The main idea of this paper is to rank Maximal Frequent Sequences (MFS) in order to identify the most important information in a text. MFS are considered as nodes of a graph in term selection step, and then are ranked in term weighting step using a graphbased algorithm. We show that the proposed method produces results superior to the-state-of-the-art methods; in addition, the best sentences were found with this method. We prove that MFS are better than other terms. Moreover, we show that the longer is MFS, the better are the results. If the stop-words are excluded, we lose the sense of MFS, and the results are worse. Other important aspect of this method is that it does not require deep linguistic knowledge, nor domain or language specific annotated corpora, which makes it highly portable to other domains, genres, and languages.

Natutal language processing: Perspective of CIC-IPN

“…For example, United States of America is a frequent sequence and thus is likely to denote a concept (which it does: a country), while United States of is a part of an almost equally frequent but longer n-gram and thus is not an MFS. We have shown that using MFSs has advantages over using single words [98].…”

Designation of Situation Model in Twitter using Maximal Frequent Sequences

“…We call an ngram frequent (more accurately, β-frequent) if it occurs more than β times in the text, where β is a predefined threshold. Frequent ngrams-we will also call them frequent sequences (FSs)-often bear important semantic meaning: they can be multiword expressions, idioms or otherwise refer to some idea important for the text [19,20]. An ngram can be a part of another, longer ngram.…”

“…Usage of Maximal Frequent Sequences helps us not only to understand the main idea of the hashtag but also to provide different kind of analysis that can be useful for marketing specialist as well as for linguist. [20] something [20] white [20] super [20] definitely [19] #follow [19] hi [19]#cybermondaymadness [19] nothing [19] wit [19] hello [18] @hashmtvstars [18] galaxys 2. #RussianMeteor 4 most frequent are spam not connected with the situation.…”

“…#RussianMeteor 4 most frequent are spam not connected with the situation. after them there are the following: [15] rt @bbcbreaking #russianmeteor shower six of the most dramatic videos @liga #meteorit @liga [13] @liga ultimas imagenes del meteorito que cayo en rusia #meteorit @hashrussianmeteor [13] at least people injured in spectacular #russianmeteor shower interior ministry @liga @liga #meteorit [5] coming up at pm @nasa experts discuss the #russianmeteor learn more about meteor listen to panel live at pm et @liga (21 words) [5] wow rt @wsj the explosion from the #russianmeteor is estimated to be as powerful as Hiroshima bombs @liga (18) [5] rt @nasa #russianmeteor is largest reported,meteor since Tunguska event impact was at utc still being @liga (17) [11] kak (how) [10] jeto (this) [ [21] ochen (very) [21] miting [20] with [20] policija (police) [20] policii (police) [20] est (is) [20] over [20] mitingujushhih (protesters) [19] rossija (Russia) [15] Navalnyj…”

“…Emotions: love (82), shit (32), super (20), fucking (12), epic (8), gorgeous (6), etc. Personalities: member (38), her (24), his (24), girl (18), anyone (17), myself (16), #together (16), mine (13), wife (12), families (11), owners (10), everybody (9), kids (8), somebody (6), men (6), etc.…”