Comment on "Clustering by Passing Messages Between Data Points"

Abstract

Frey and Dueck (Reports, 16 February 2007, p. 972) described an algorithm termed "affinity propagation" (AP) as a promising alternative to traditional data clustering procedures. We demonstrate that a well-established heuristic for the p-median problem often obtains clustering solutions with lower error than AP and produces these solutions in comparable computation time.F rey and Dueck (1) described an algorithm for analyzing complex data sets termed "affinity propagation" (AP). The algorithm extracts a subset of representative objects or "exemplars" from the complete object set by exchanging real-valued messages between data points. Clusters are formed by assigning each data point to its most similar exemplar. The authors reported that "[a]ffinity propagation found clusters with much lower error than other methods, and it did so in less than one-hundredth the amount of time" (1). We demonstrate that an efficient implementation of a 40-year-old heuristic for the well-known p-median model (PMM) often provides lower-error solutions than AP in comparable central processing unit (CPU) time.For consistency with AP in (1), we present the PMM as a sum of similarities maximization problem, while recognizing that this is equivalent to the more common form of minimizing the sum of dissimilarities (e.g., distances or costs). The PMM is a general mathematical problem that can be concisely stated as follows: Given an m × n similarity matrix, S, select p columns from S such that the sum of the maximum values within each row of the selected columns is maximized (2). Thus, each row is effectively assigned to its most similar selected column (exemplar) with the goal of maximizing overall similarity. One classic example of the PMM occurs in facility location planning: Locate p plants such that the total distance (or cost) required to serve m demand points is minimized. In data analysis applications where S is an n × n matrix of negative squared Euclidean distances between objects, clustering the n objects using the PMM corresponds to the selection of p exemplars to minimize error, which is defined as the sum of the squared Euclidean distances of each object to its nearest exemplar.Lagrangian relaxation methods enable the exact solution of PMM instances with n ≤ 500 objects (3, 4). For larger problems, a vertex substitution heuristic (VSH) developed in (5) has been the standard for comparison for nearly four decades. The VSH begins with the random selection of a subset of p exemplars, which is iteratively refined by evaluating the effects of substituting an unselected point for one of the selected exemplars. Frey and Dueck assert that this type of strategy "works well only when the number of clusters is small and chances are good that at least one random initialization is close to a good solution" (1). To the contrary, the VSH is remarkably effective and is often the engine for metaheuristics such as tabu search (6) and variable neighborhood search (7).We compared AP to an efficient implementation of VSH (7) across eight data set...

Text Clustering with Seeds Affinity Propagation

“…The most significant advantage of SAP is that it is better than k-means in the foregoing evaluations while k-means runs 200 times (the best run is used to compare with SAP) and costs about twenty-fold of SAP in time. This result also confirms the one in [7], [30], and [31]. Furthermore, even after 10000 runs of k-means -with a size of 400 documents (F-measure: 0.406; Entropy: 0.677), we can't get similar results as SAP.…”

“…Many detailed analysis of the AP approach have been carried out (see for instance [30] and [31]) for various datasets with different scales. These studies show that for small datasets, there are only minor differences between traditional strategies (such as p-median model and vertex substitution heuristic) and Affinity Propagation clustering for both precision and CPU execution time.…”

“…500 clusters. Moreover, in [7][8][9][10][11][12] and [30][31][32][33], it has been identified that the similarity measurement has a great influence on AP clustering.…”

Exemplar-Based Clustering via Simulated Annealing

“…Moreover, for consistency with Dueck (2007, 2008) and Brusco and Köhn (2008a), we present the p-median problem as one of similarity maximization, as opposed to the equivalent dissimilarity minimization version that is also common in the literature (Avella et al, 2007;Mulvey & Crowder, 1979;Resende & Werneck, 2004). We denote S = [s ij ] as an N × N nonpositive similarity matrix that is measured for the set of N objects indexed by the set I = {1, 2, .…”

“…Brusco and Köhn (2008a) presented an adaptation of Hansen and Mladenović's (1997) fast VSH procedure for MPMP that incorporates the preference information in the objective function. This program, vsh_fc.m, is available as a MATLAB m-file from the website http://mailer.fsu.edu/~mbrusco.…”