High‐speed rough clustering for very large document collections

Abstract

Document clustering is an important tool, but itis not yet widely used in practice probably because of its high computational complexity. This article explores techniques of high-speed rough clustering of documents, assuming that it is sometimes necessary to obtain a clustering result in a shorter time, although the result is just an approximate outline of document clusters. A promising approach for such clustering is to reduce the number of documents to be checked for generating cluster vectors in the leader-follower clustering algorithm. Based on this idea, the present article proposes a modified Crouch algorithm and incomplete single-pass leaderfollower algorithm. Also, a two-stage grouping technique, in which the first stage attempts to decrease the number of documents to be processed in the second stage by applying a quick merging technique, is developed. An experiment using a part of the Reuters corpus RCV1 showed empirically that both the modified Crouch and the incomplete single-pass leader-follower algorithms achieve clustering results more efficiently than the original methods, and also improved the effectiveness of clustering results. On the other hand, the two-stage grouping technique did not reduce the processing time in this experiment.

Informational Paradigm, management of uncertainty and theoretical formalisms in the clustering framework: A review

“…Atanassov [21,22,25] , Atanassov and Gargov [23] and Xu [314] . [145] Herawan et al [125] In the literature, there has been a lively debate by some scholars on the appropriateness of the term intuitionistic fuzzy sets (IFS) adopted by Atanassov for his theory. Grzegorzewski and Mròwka [116] suggested a list of possible alternative terms that would however allow one to retain the acronym IFS: incomplete fuzzy sets, inaccurate fuzzy sets, imperfect fuzzy sets, indefinite fuzzy sets, indeterminate fuzzy sets, indistinct fuzzy sets.…”

Visualizing the world's scientific publications

“…For example, the K ‐means clustering algorithm takes a long time to compute, requires a priori knowledge regarding the number of clusters, and is sensitive to the choice of initial cluster centers. Several algorithms have been proposed to reduce computation time in cluster analysis by using parallel computation techniques or at the cost of the clustering results (Fernandez & Gomez, ; Kishida, ). Recently, we developed a minimum span clustering (MSC) method for clustering complex networks.…”

Visualizing the clustering of financial networks and profitability of stocks

“…Moreover, HC involves the calculation of similarities between all pairs of components for each clustering level, and is not efficient for handling large data sets. Several algorithms have been proposed for reducing computation time in cluster analysis by using parallel computation techniques or at the cost of the clustering results [2,10,11]. However, most clustering algorithms still require pre-given assumptions as inputs for the classification of complex systems, including the number of clusters, cluster sizes and boundary conditions.…”