A New LSA and Entropy-Based Approach for Automatic Text Document Summarization

Yadav, C. S.; Sharan, Aditi

doi:10.4018/ijswis.2018100101

Cited by 16 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Extractive summarization has been applied to several domains such as automatic highlighting of text (Grootjen & Kachergis, 2018), web articles, multi-document summarization (Gupta & Siddiqui, 2012;Wang et al, 2008) and several others. A number of diversified techniques have been employed for extractive summarization such as statistical-based approaches (Ferreira et al, 2013), genetic algorithm (Sim on et al, 2018, graph-based (Erkan & Radev, 2004;Mihalcea & Tarau, 2004), neural networks (Sinha et al, 2018), optimization-based (Gillick et al, 2008;McDonald, 2007), semantic similarity-based (Gong & Liu, 2001;Steinberger & Ježek, 2004;Yadav & Sharan, 2018), fuzzy-logic based (Abbasi-ghalehtaki et al, 2016;Valladares-vald et al, 2020) and centroid-based techniques (Radev et al, 2004). This work focuses on the previous research on the unsupervised approach.…”

Section: Extractive Summarizationmentioning

confidence: 99%

Extractive summarization using concept‐space and keyword phrase

2022

View full text Add to dashboard Cite

PROBLEM: Text summarization is widely used to extract relevant information from documents. The extensive unstructured data are generated in various domains, developing a concise and coherent form of information has become an inevitable and assertive problem among the research community. Automatic text summarization plays a crucial role in acquiring potent knowledge effectively and efficiently for each domain of information. Medical data contains crucial information regarding human diseases and their symptoms. The proposed method will benefit the research community and health experts as it saves a lot of time and resources to compute the patients' summary during diagnosis. APPROACH: In this research, an unsupervised approach is proposed for extractive summarization, based on semantic similarity and keyword-phrase extraction. Merging Concept Map and the RAKE method, a generic summary is computed based on threshold values. Both single-document and multi-document summarization can be accomplished with the approach. RESULTS: To evaluate the proposed unsupervised approach, various biomedical transcripts of neuro-science, general medicine, gastroenterology, orthopaedic and radiology domain are used. MT Sample Dataset is used to collect 1040 different transcripts. Using the proposed approach, an average ROUGE score of 0.77 for single-document summarization; however, for generic summary, an average ROUGE is 0.72. The proposed technique is validated for the previous corpus of BioMed articles, and results are better with state-of-the-art techniques.

show abstract

Section: Extractive Summarizationmentioning

confidence: 99%

Extractive summarization using concept‐space and keyword phrase

2022

View full text Add to dashboard Cite

show abstract

“…Since high entropy of a sentence implies more coverage, the method has inherent bias towards long sentences, favoring their inclusion in summary. Yadav & Sharan (2018) gauge entropy in latent semantic space of the document by computing probability distribution of topics in sentences and vice-versa. They use Latent Semantic Analysis (LSA) to reveal the latent semantic space of the document.…”

Section: Entropy For Document Summarizationmentioning

confidence: 99%

Investigating Entropy for Extractive Document Summarization

Khurana,

Bhatnagar

2021

Preprint

View full text Add to dashboard Cite

Automatic text summarization aims to cut down readers' time and cognitive effort by reducing the content of a text document without compromising on its essence. Ergo, informativeness is the prime attribute of document summary generated by an algorithm, and selecting sentences that capture the essence of a document is the primary goal of extractive document summarization.In this paper, we employ Shannon's entropy to capture informativeness of sentences. We employ Non-negative Matrix Factorization (NMF) to reveal probability distributions for computing entropy of terms, topics, and sentences in latent space. We present an information theoretic interpretation of the computed entropy, which is the bedrock of the proposed E-Summ algorithm, an unsupervised method for extractive document summarization. The algorithm systematically applies information theoretic principle for selecting informative sentences from important topics in the document. The proposed algorithm is generic and fast, and hence amenable to use for summarization of documents in real time. Furthermore, it is domain-, collection-independent and agnostic to the language of the document. Benefiting from strictly positive NMF factor matrices, E-Summ algorithm is transparent and explainable too.We use standard ROUGE toolkit for performance evaluation of the proposed method on four well known public data-sets. We also perform quantitative as- *

show abstract

“…It has been used in several areas of applications such as web articles [2], meeting and telephonic conversations [3], multi-document summarization [1], [38], [39], automatic highlighting of text [40] and many others. To produce an extractive summary various technique are used such as genetic algorithm [41], conditional random fields [42], neural networks [43], semantic similarity such as latent semantic analysis [44]- [46]. Along with these supervised learning techniques, unsupervised learning methods such as fuzzy logic [28], [29], [47], [48], k-means clustering along with term frequency-inverse document frequency [49] has been used.…”

Section: A Extractive Summarizationmentioning

confidence: 99%

Automatic Keyword and Sentence-Based Text Summarization for Software Bug Reports

Jindal

Kaur

2020

IEEE Access

View full text Add to dashboard Cite

Text Summarization is a process which efficiently retrieves the relevant information from documents. The objective of the proposed, unsupervised approach is to summarize bug reports (software artefacts) with complete content and diversified information. The proposed approach utilizes Rapid Automatic Keyword Extraction and term frequency-inverse document frequency method to extract meaningful keywords and key-phrases with a relevant score. For sentence extraction, fuzzy C-means clustering is used to extracts sentences having high degree of membership from each cluster above a set threshold value. A ruleengine is used for sentence selection. The rules are generated with the domain knowledge and based on the extracted information by the keywords and sentences selected by the clustering method. Cohesive and coherent summary is generated by the proposed method on apache bug reports. For redundancy removal and to re-rank generated summary, hierarchical clustering is presented to enrich the extracted summary. The proposed approach is evaluated on newly constructed Apache project Bug Report Corpus (APBRC) and existing Bug Report Corpus (BRC). The results are compared on the basis of performance metrics such as precision, recall, pyramid precision and F-score. The experimental results depict that our proposed approach attains significant improvement over other baseline approaches such as BRC and LRCA. It also attains significant improvement over existing state-of-art unsupervised approaches such as Hurried, centroid and others. It extracts significant keyword phrases and sentences from each cluster to achieve full coverage and coherent summary. The results evaluated on APBRC corpus attains an average value of 78.22%, 82.18%, 80.10% and 81.66% for precision, recall, f-score and pyramid precision respectively. INDEX TERMS Text summarization, rapid automatic keyword extraction, fuzzy c-means, hierarchical clustering, bug reports, rule engine.

show abstract

A New LSA and Entropy-Based Approach for Automatic Text Document Summarization

Cited by 16 publications

References 54 publications

Extractive summarization using concept‐space and keyword phrase

Extractive summarization using concept‐space and keyword phrase

Investigating Entropy for Extractive Document Summarization

Automatic Keyword and Sentence-Based Text Summarization for Software Bug Reports

Contact Info

Product

Resources

About