Efficient stochastic algorithms for document clustering

Abstract

Clustering has become an increasingly important and highly complicated research area for targeting useful and relevant information in modern application domains such as the World Wide Web. Recent studies have shown that the most commonly used partitioning-based clustering algorithm, the K-means algorithm, is more suitable for large datasets. However, the K-means algorithm may generate a local optimal clustering. In this paper, we present novel document clustering algorithms based on the Harmony Search (HS) optimization method. By modeling clustering as an optimization problem, we first propose a pure HS based clustering algorithm that finds near-optimal clusters within a reasonable time. Then, harmony clustering is integrated with the K-means algorithm in three ways to achieve better clustering by combining the explorative power of HS with the refining power of the K-means. Contrary to the localized searching property of K-means algorithm, the proposed algorithms perform a globalized search in the entire solution space. Additionally, the proposed algorithms improve K-means by making it less dependent on the initial parameters such as randomly chosen initial cluster centers, therefore, making it more stable. The behavior of the proposed algorithm is theoretically analyzed by modeling its population variance as a Markov chain. We also conduct an empirical study to determine the impacts of various parameters on the quality of clusters and convergence behavior of the algorithms. In the experiments, we apply the proposed algorithms along with K-means and a Genetic Algorithm (GA) based clustering algorithm on five different document datasets. Experimental results reveal that the proposed algorithms can find better clusters and the quality of clusters is comparable based on F-measure, Entropy, Purity, and Average Distance of Documents to the Cluster Centroid (ADDC).

Document Clustering Based on Firefly Algorithm

“…The smaller value of ADDC is more compact clustering solution (Forsati et al, 2013). Figure 9 illustrates the quality performance metrics; F-measure, Entropy, Purity and ADDC results between the WFA and WFA II .…”

“…Term FrequencyInverse Document Frequency (TF-IDF) is a technique that has been widely used to represent documents in the form of numerical weights in the vector space (Manning et al, 2008;Forsati et al, 2013). TF-IDF for each term in a document is equal to the term frequency multiply by the inverse documents frequency, idf, which can be calculated using Equation 5 (Manning et al, 2008):…”

“…The Average Distance between Documents and Cluster centroid (ADDC) (Cui et al, 2005;Forsati et al, 2013) …”

“…On the other hand, the bottom up method starts from single clusters that contain a single item. The clusters are later merged based on an identified similarity measures (Jain et al, 1999;Forsati et al, 2013). The UPGMA is popular algorithm of such approach (Yujian and Liye, 2010).…”

“…Even though K-means is a simple method and is efficient for large dataset, it suffers from the local optima problem (Forsati et al, 2013).…”