Scalable k-means++

Abstract

Over half a century old and showing no signs of aging, k-means remains one of the most popular data processing algorithms. As is well-known, a proper initialization of k-means is crucial for obtaining a good final solution. The recently proposed k-means++ initialization algorithm achieves this, obtaining an initial set of centers that is provably close to the optimum solution. A major downside of the k-means++ is its inherent sequential nature, which limits its applicability to massive data: one must make k passes over the data to find a good initial set of centers. In this work we show how to drastically reduce the number of passes needed to obtain, in parallel, a good initialization. This is unlike prevailing efforts on parallelizing k-means that have mostly focused on the post-initialization phases of k-means. We prove that our proposed initialization algorithm k-means| | obtains a nearly optimal solution after a logarithmic number of passes, and then show that in practice a constant number of passes suffices. Experimental evaluation on realworld large-scale data demonstrates that k-means| | outperforms k-means++ in both sequential and parallel settings.

MR-RBAT: Anonymizing Large Transaction Datasets Using MapReduce

“…Chierichetti et al [6] implemented an existing greedy Maxk-cover algorithm using MapReduce efficiently and achieved provably approximation to sequential results. Bahmani et al [2] obtained a parallel implementation of K-means++ [1] and empirically showed to have achieved similar results in a constant number of rounds. MapReduce solutions have also been proposed for anonymization [20,22,21], but were limited to achieving k-anonymity for relational data only.…”

“…Note that map output with the same key must be sent to the same reducer, so the number of reducers needed is determined by min(R, (2). The reduce cost t R of a single MapReduce round is dominated by the cost of setting up the reducers and reading the shuffled data sent by the mappers:…”

Evaluating the Use of Clustering for Automatically Organising Digital Library Collections

“…Using the LDA document-cluster distribution instead of TFIDF leads to a significant reduction in processing time, however the quality of the resulting model suffers and is lower than the pure LDA results. K-Means is thus only viable for smaller collections, although the exact limit depends on what optimisations [25] can be achieved through improved initialisation [4] or parallelisation [32].…”

Using data to build a better EM: EM* for big data

“…Chief among these is that traditional ways of processing data have become inadequate. This is witnessed in how the dozen or so traditional algorithms [2], e.g., k-means or EM, have begun to be rethought and retooled [3][4][5] to deal with data that is now too massive, too complex, produced too quickly, etc., to be effectively analyzed as we have in the past.…”