A Frequent Concepts Based Document Clustering Algorithm

Abstract

This paper presents a novel technique of document clustering based on frequent concepts. The proposed technique, FCDC (Frequent Concepts based document clustering), a clustering algorithm works with frequent concepts rather than frequent items used in traditional text mining techniques. Many well known clustering algorithms deal with documents as bag of words and ignore the important relationships between words like synonyms. the proposed FCDC algorithm utilizes the semantic relationship between words to create concepts. It exploits the WordNet ontology in turn to create low dimensional feature vector which allows us to develop a efficient clustering algorithm. It uses a hierarchical approach to cluster text documents having common concepts. FCDC found more accurate, scalable and effective when compared with existing clustering algorithms like Bisecting K-means , UPGMA and FIHC.

Semantic based Document Clustering: A Detailed Review

“…In [11], a new technique based on frequent concepts for document clustering is proposed. Frequent Concepts based Document Clustering (FCDC) algorithm utilizes the semantic relationship between words, explored using WordNet ontology, to create concepts.…”

“…But, taking into account synonyms and hypernyms, disambiguated only by PoS tags, is not successful in improving clustering effectiveness because of the noise produced by all the incorrect senses extracted from WordNet. A possible solution is proposed which uses a word-by-word disambiguation in order to choose the correct sense of a word in [11]. In [6] Clustering based on Frequent Word Sequences (CFWS) has been proposed.…”

Semantic Sentiment Analysis of Arabic Texts

“…The experiments showed that using the semantic WN concepts features were promising and outperformed the baseline BoW model. Also, Baghel and Dhir in [18] proposed a hierarchy clustering algorithm to cluster the documents based on the concepts representation. The concepts were extracted from WN using the FstC WSD strategy.…”

Clustering Web Documents based on Efficient Multi-Tire Hashing Algorithm for Mining Frequent Termsets

“…A good stemmer should be able to convert different syntactic forms of a word into its normalized form, reduce the number of index terms, save memory and storage and may increase the performance of clustering algorithms to some extent; meanwhile it should try stemming. Porter Stemmer [27] is a widely applied method to stem documents. It is compact, simple and relatively accurate.…”