Clustering data streams: theory and practice

Guha, Sudipto; Meyerson, Adam; Mishra, Nina; Motwani, Rajeev; O'Callaghan, Liadan

doi:10.1109/tkde.2003.1198387

Cited by 706 publications

(431 citation statements)

References 72 publications

Supporting

Mentioning

391

Contrasting

Unclassified

Order By: Relevance

“…no knowledge of the number of clusters), (2) tracking cluster evolutions, (3) speed and complexity of computation (including highdimenstionality) and (4) outliers, to name a few. There are several papers that review these challenges and some of the algorithms designed to tackle them (Yogita and Toshniwal, 2012;Khalilian and Mustapha, 2010;Guha et al, 2003).…”

Section: Clustering Algorithmsmentioning

confidence: 99%

Hierarchical Adaptive Means (HAM) clustering for hardware-efficient, unsupervised and real-time spike sorting

Paraskevopoulou

Eftekhar

et al. 2014

Journal of Neuroscience Methods

View full text Add to dashboard Cite

This work presents a novel unsupervised algorithm for real-time adaptive clustering of neural spike data (spike sorting). The proposed Hierarchical Adaptive Means (HAM) clustering method combines centroid-based clustering with hierarchical cluster connectivity to classify incoming spikes using groups of clusters. It is described how the proposed method can adaptively track the incoming spike data without requiring any past history, iteration or training and autonomously determines the number of spike classes. Its performance (classification accuracy) has been tested using multiple datasets (both simulated and recorded) achieving a near-identical accuracy compared to k-means (using 10-iterations and provided with the number of spike classes). Also, its robustness in applying to different feature extraction methods has been demonstrated by achieving classification accuracies above 80% across multiple datasets. Last but crucially, its low complexity, that has been quantified through both memory and computation requirements makes this method hugely attractive for future hardware implementation.

show abstract

Section: Clustering Algorithmsmentioning

confidence: 99%

Hierarchical Adaptive Means (HAM) clustering for hardware-efficient, unsupervised and real-time spike sorting

Paraskevopoulou

Eftekhar

et al. 2014

Journal of Neuroscience Methods

View full text Add to dashboard Cite

show abstract

“…We derive the idea of dividing data into chunks and working on each of them separately by the work of Guha et al [4]. The easiest way to divide the data is based on the timestamp (creating a new chunk for each interval of time).…”

Section: Dividing Data Into Chunksmentioning

confidence: 99%

DS-Means: Distributed Data Stream Clustering

Guerrieri

Montresor

2012

Euro-Par 2012 Parallel Processing

View full text Add to dashboard Cite

Abstract. This paper proposes DS-means, a novel algorithm for clustering distributed data streams. Given a network of computing nodes, each of them receiving its share of a distributed data stream, our goal is to obtain a common clustering under the following restrictions (i) the number of clusters is not known in advance and (ii) nodes are not allowed to share single points of their datasets, but only aggregate information. A motivating example for DS-means is the decentralized detection of botnets, where a collection of independent ISPs may want to detect common threats, but are unwilling to share their precious users' data. In DS-means, nodes execute a distributed version of K-means on each chunk of data they receive to provide a compact representation of the data of the entire network. Later, X-means is executed on this representation to obtain an estimate of the number of clusters. A number of experiments on both synthetic and real-life datasets show that our algorithm is precise, efficient and robust.

show abstract

“…A data stream is represented by S = {o 1 to represent a unit, only one value, a unit identifier, is used in this paper. For instance, an object o k is transformed to a unit…”

Section: A Cluster Streamingmentioning

confidence: 99%

“…Recently, many data mining methods [1] for a data stream have been actively introduced. A data stream is an ordered sequence of objects o 1 , …, o n that must be accessed in order and that can be read only once or a small specified number of times.…”

Section: Introductionmentioning

confidence: 99%

Anomaly Intrusion Detection based on Clustering a Data Stream

Jinsuk¹,

Oh²

2012

IJFCC

View full text Add to dashboard Cite

Abstract-This paper proposes a new clustering algorithm which continuously models a data stream. A set of features is used to represent the characteristics of an activity. For each feature, the clusters of feature values corresponding to activities observed so far in an audit data stream are identified by the proposed clustering algorithm for data streams. As a result, without maintaining any historical activity of a user physically, new activities of the user can be continuously reflected to the on-going result of clustering.

show abstract

Clustering data streams: theory and practice

Cited by 706 publications

References 72 publications

Hierarchical Adaptive Means (HAM) clustering for hardware-efficient, unsupervised and real-time spike sorting

Hierarchical Adaptive Means (HAM) clustering for hardware-efficient, unsupervised and real-time spike sorting

DS-Means: Distributed Data Stream Clustering

Anomaly Intrusion Detection based on Clustering a Data Stream

Contact Info

Product

Resources

About