Single Pass Fuzzy C Means

Hore, Prodip; Hall, Lawrence O.; Goldgof, Dmitry

doi:10.1109/fuzzy.2007.4295372

Cited by 102 publications

(94 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The quality difference and speed up obtained on this 75 million example data set was excellent. There are similar good results available for other data sets [18].…”

Section: Methodssupporting

confidence: 74%

See 1 more Smart Citation

Creating Streaming Iterative Soft Clustering Algorithms

Hore

Hall

Goldgof

2007

NAFIPS 2007 - 2007 Annual Meeting of the North American Fuzzy Information Processing Society

Self Cite

View full text Add to dashboard Cite

Abstract-There are an increasing number of large labeled and unlabeled data sets available. Clustering algorithms are the best suited for helping one make sense out of unlabeled data. However, scaling iterative clustering algorithms to large amounts of data has been a challenge. The computation time can be very great and for data sets that will not fit in even the largest memory, only carefully chosen subsets of data can be practically clustered. We present a general approach which enables iterative fuzzy/possibilistic clustering algorithms to be turned into algorithms that can handle arbitrary amounts of streaming data. The computation time is also reduced for very large data sets while the results of clustering will be very similar to clustering with all the data, if that was possible. We introduce transformed equations for fuzzy-c-means, possibilistic c-means, the Gustafson-Kessel algorithm and show the excellent performance with a streaming fuzzy c-means implementation. The resulting clusters are both sensible and for comparable data sets (those that fit in memory) almost identical to those obtained with the original clustering algorithm.

show abstract

“…The quality difference and speed up obtained on this 75 million example data set was excellent. There are similar good results available for other data sets [18].…”

Section: Methodssupporting

confidence: 74%

“…There has been research on clustering large or very large data sets [13], [14], [15], [16], [17], [18]. Birch [13] is a data clustering method for large data sets.…”

Section: Related Workmentioning

confidence: 99%

Creating Streaming Iterative Soft Clustering Algorithms

Hore

Hall

Goldgof

2007

NAFIPS 2007 - 2007 Annual Meeting of the North American Fuzzy Information Processing Society

Self Cite

View full text Add to dashboard Cite

show abstract

“…Popular incremental fuzzy clustering algorithms for data streams include single-pass FCM [27] and online FCM (OFCM) [28]. Both process data chunk by chunk (by-pattern) and estimate centroids for entire data set by extracting summary information from each chunk, but the ways they handle chunks are different.…”

Section: Previous Work In the Fields Of Data Streammentioning

confidence: 99%

Tracking Time Evolving Data Streams for Short-Term Traffic Forecasting

2017

View full text Add to dashboard Cite

Data streams have arisen as a relevant topic during the last few years as an efficient method for extracting knowledge from big data. In the robust layered ensemble model (RLEM) proposed in this paper for shortterm traffic flow forecasting, incoming traffic flow data of all connected road links are organized in chunks corresponding to an optimal time lag. The RLEM model is composed of two layers. In the first layer, we cluster the chunks by using the Graded Possibilistic c-Means method. The second layer is made up by an ensemble of forecasters, each of them trained for short-term traffic flow forecasting on the chunks belonging to a specific cluster. In the operational phase, as a new chunk of traffic flow data presented as input to the RLEM, its memberships to all clusters are evaluated, and if it is not recognized as an outlier, the outputs of all forecasters are combined in an ensemble, obtaining in this a way a forecasting of traffic flow for a short-term time horizon. The proposed RLEM model is evaluated on a synthetic data set, on a traffic flow data simulator and on two real-world traffic flow data sets. The model gives an accurate forecasting of the traffic flow rates with outlier detection and shows a good adaptation to nonstationary traffic regimes. Given its characteristics of outlier detection, accuracy, and robustness, RLEM can be fruitfully integrated in traffic flow management systems.

show abstract

“…3.1) for X [t] ; obtain the codeword matrix B and coefficient matrix D. 5: Run weighted k-means (Sect. 3.2) on X [t] and set the weights by (14); obtain X…”

Section: Incremental Importance Samplingmentioning

confidence: 99%

“…Similar idea has been adopted for soft clustering, where each datum could belong to multiple clusters with different membership degrees. In [14], previous data are categorized into several soft clusters, and each cluster is summarized as its center point with the weight equal to sum of the membership degrees of the data in this cluster. Then, the weighted fuzzy c-means algorithm is adopted to find clusters for the combination of the weighted samples and the current data.…”

Section: Introductionmentioning

confidence: 99%

Single-Pass K-SVD for Efficient Dictionary Learning

Chang

Lin

Chen

et al. 2013

Circuits Syst Signal Process

View full text Add to dashboard Cite

Sparse representation has been widely used in machine learning, signal processing and communications. K-SVD, which generalizes k-means clustering, is one of the most famous algorithms for sparse representation and dictionary learning. K-SVD is an iterative method that alternates between encoding the data sparsely by using the current dictionary and updating the dictionary based on the sparsely represented data. In this paper, we introduce a single-pass K-SVD method. In this method, the previous input data are first summarized as a condensed representation of weighted samples. Then, we developed a weighted K-SVD algorithm to learn a dictionary from the union of this representation and the newly input data. Experimental results show that our approach can approximate K-SVD's performance well by consuming considerably less storage resource.

show abstract

Single Pass Fuzzy C Means

Cited by 102 publications

References 23 publications

Creating Streaming Iterative Soft Clustering Algorithms

Creating Streaming Iterative Soft Clustering Algorithms

Tracking Time Evolving Data Streams for Short-Term Traffic Forecasting

Single-Pass K-SVD for Efficient Dictionary Learning

Contact Info

Product

Resources

About