Tackling redundancy in text summarization through different levels of language analysis

Lloret, Elena; Palomar, Manuel

doi:10.1016/j.csi.2012.08.001

Cited by 23 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The text summarization can be understood by Fig. 1 where the input represents a large amount of data and the output is condensed useful information [1][3] [17]. Text summarization is employed in various elds such as in research, in document retrieval, in smart handling of technology etc.…”

Section: Introductionmentioning

confidence: 99%

WITHDRAWN: Refining Readability by Tackling Redundancy and Time through Modified Rule based Extractive Summarization Approach for Low Resource Indian Languages-Hindi andPunjabi

Sachdeva

Jhatta²,

Bathla³

2022

Preprint

View full text Add to dashboard Cite

A rule based extractive text summarization system is proposed to obtain comprehensible summarized text of input Hindi and Punjabi text documents. The input document to be summarized can be any handwritten text or any text from internet saved in .rtf format. It works on Hindi and Punjabi documents and summarizes text of any length i.e. system summarizes text irrespective of size. The extensive rule based approach is applied consisting preprocessing and processing phases. Each phase consists of rules which are to be followed to get the desired concise summary. In the preprocessing phase, rules of Tokenization, Stop word removal and segmentation are applied whereas in processing phase rules such as Monetary values, Measurement Values, Special Symbols, Equations, Figure Number in sentences are applied. Also, other features such as frequently occurring words and maximum length word are added to make the system output comprehendible. The system is made user friendly by proposing a graphical user interface (GUI). The generated summary by the proposed system is compared with the summary of documents generated by human experts and is evaluated in terms of parameters such as Compression Ratio and Execution Time. Further, accuracy is calculated based on Precision, Recall and F-score. It has been observed that the compression ratio obtained lies in the range of 39–50% for 200, 300 and 400 words paragraphs. The execution time is quite less which do varies with the size of the paragraph entered as the summary size varies. In general, it has been observed that the execution time for 200 words paragraph takes 1519.147 milliseconds, for 300 words paragraph the time is 3022.825 milliseconds whereas for 400 words size paragraph, the time taken is 4023.123 milliseconds. The time is quite less as compared to human generated summary. The accuracy obtained in each case is above 95% which is high as compared to humans and other state of art methods. This shows that the system is robust enough and provides definitive results.

show abstract

Section: Introductionmentioning

confidence: 99%

WITHDRAWN: Refining Readability by Tackling Redundancy and Time through Modified Rule based Extractive Summarization Approach for Low Resource Indian Languages-Hindi andPunjabi

Sachdeva

Jhatta²,

Bathla³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…They use a greedy algorithm (Wan et al (2007), Zhang et al (2005)) to impose a diversity penalty on the sentences. Lloret and Palomar (2013) have used three different levels of language analysis to tackle redundancy in text summarization. Different levels of language analysis are: lexical, syntactic and semantic.…”

Section: Machine Learning Techniques Applied Inmentioning

confidence: 99%

Query-Based Extractive Text Summarization Using Sense-Oriented Semantic Relatedness Measure

Rahman,

Borah

2023

Arab J Sci Eng

View full text Add to dashboard Cite

This paper presents a query-based extractive text summarization method by using sense-oriented semantic relatedness measure. We have proposed a Word Sense Disambiguation (WSD) technique to find the exact sense of a word present in the sentence. It helps in extracting query relevance sentences while calculating the sense-oriented sentence semantic relatedness score between the query and input text sentence. The proposed method uses five unique features to make clusters of query-relevant sentences. A redundancy removal technique is also put forward to eliminate redundant sentences. We have evaluated our proposed WSD technique with other existing methods by using Senseval and SemEval datasets. Experimental evaluation and discussion signifies the better performance of proposed WSD method over current systems in terms of F-score. We compare our proposed query-based extractive text summarization method with other methods participated in Document Understanding Conference (DUC) and as well as with current methods. Evaluation and comparison state that the proposed query-based extractive

show abstract

“…While extractive methods tend to have better performance on the first two aspects, they are typically less coherent and more redundant than abstractive ones, where new sentences are often generated by sentence fusion and compression, which helps detecting and removing redundancy (Lebanoff et al, 2019). Although eliminating redundancy has been initially and more intensely studied in the field of multidocument summarization (Lloret and Sanz, 2013), because important sentences selected from multiple documents (about the same topic) are more likely to be redundant than sentences from the same document, generating a non-redundant summary should still be one of the goals for single document summarization (Lin et al, 2009).…”

Section: Introductionmentioning

confidence: 99%

Systematically Exploring Redundancy Reduction in Summarizing Long Documents

Xiao¹,

Carenini²

2020

Preprint

View full text Add to dashboard Cite

Our analysis of large summarization datasets indicates that redundancy is a very serious problem when summarizing long documents. Yet, redundancy reduction has not been thoroughly investigated in neural summarization. In this work, we systematically explore and compare different ways to deal with redundancy when summarizing long documents. Specifically, we organize the existing methods into categories based on when and how the redundancy is considered. Then, in the context of these categories, we propose three additional methods balancing non-redundancy and importance in a general and flexible way. In a series of experiments, we show that our proposed methods achieve the state-of-the-art with respect to ROUGE scores on two scientific paper datasets, Pubmed and arXiv, while reducing redundancy significantly. 1

show abstract

Tackling redundancy in text summarization through different levels of language analysis

Cited by 23 publications

References 13 publications

WITHDRAWN: Refining Readability by Tackling Redundancy and Time through Modified Rule based Extractive Summarization Approach for Low Resource Indian Languages-Hindi andPunjabi

WITHDRAWN: Refining Readability by Tackling Redundancy and Time through Modified Rule based Extractive Summarization Approach for Low Resource Indian Languages-Hindi andPunjabi

Query-Based Extractive Text Summarization Using Sense-Oriented Semantic Relatedness Measure

Systematically Exploring Redundancy Reduction in Summarizing Long Documents

Contact Info

Product

Resources

About