Fault tolerant decentralised -Means clustering for asynchronous large-scale networks

Abstract

The K-Means algorithm for cluster analysis is one of the most influential and popular data mining methods. Its straightforward parallel formulation is well suited for distributed memory systems with reliable interconnection networks, such as massively parallel processors and clusters of workstations. However, in large-scale geographically distributed systems the straightforward parallel algorithm can be rendered useless by a single communication failure or high latency in communication paths. The lack of scalable and fault tolerant global communication and synchronisation methods in large-scale systems has hindered the adoption of the K-Means algorithm for applications in large networked systems such as wireless sensor networks, peer-to-peer systems and mobile ad hoc networks. This work proposes a fully distributed K-Means algorithm (Epidemic K-Means) which does not require global communication and is intrinsically fault tolerant. The proposed distributed K-Means algorithm provides a clustering solution which can approximate the solution of an ideal centralised algorithm over the aggregated data as closely as desired. A comparative performance analysis is carried out against the state of the art sampling methods and shows that the proposed method overcomes the limitations of the sampling-based approaches for skewed clusters distributions. The experimental analysis confirms that the proposed algorithm is very accurate and fault tolerant under unreliable network conditions (message loss and node failures) and is suitable for asynchronous networks of very large and extreme scale.

Designing a Synchronization-reducing Clustering Method on Manycores

“…Fatta et al designed a fault tolerant epidemic clustering algorithm which does not require global communication [7]. However, their research is focused on handling network failures and does not provide the same SSE cost as the standard k-means clustering algorithm.…”

Agreement in Epidemic Information Dissemination

“…Correctly operating processes reach consensus when all of them detect the failed ones. In [14], an Epidemicbased aggregation protocol is used to perform a global synchronisation and reduction operation for a fully decentralised K-Means clustering without global communication patterns under node churn and message loss. The work in [15] investigated heuristic methods to detect the convergence of Epidemic aggregation and those methods could be used to build a consensus protocol for data aggregation.…”

GDCluster: A General Decentralized Clustering Algorithm

“…Eyal et al [31] provide a generic algorithm for clustering in a static network. Fatta et al [32] propose a gossip-based distributed k-means clustering, which is initiated with similar initial centroids, and proceeds towards centroid convergence with rounds of gossiping. Shen et al [33] propose a distributed clustering in a static network, incorporating information theory measures.…”