DCF: An Efficient Data Stream Clustering Framework for Streaming Applications

Abstract

Abstract. Streaming applications, such as environment monitoring and vehicle location tracking require handling high volumes of continuously arriving data and sudden fluctuations in these volumes while efficiently supporting multidimensional historical queries. The use of the traditional database management systems is inappropriate because they require excessive number of disk I/O in continuously updating massive data streams. In this paper, we propose DCF (Data Stream Clustering Framework), a novel framework that supports efficient data stream archiving for streaming applications. DCF can reduce a great amount of disk I/O in the storage system by grouping incoming data into clusters and storing them instead of raw data elements. In addition, even when there is a temporary fluctuation in the amount of incoming data, it can stably support storing all incoming raw data by controlling the cluster size. Our experimental results show that our approach significantly reduces the number of disk accesses in terms of both inserting and retrieving data.

A fuzzy c means variant for clustering evolving data streams

“…However, density based algorithms are different from fuzzy clustering algorithms as they try to optimize a different objective function. In [21] a framework for efficiently archiving high volumes of streaming data was proposed, which reduces disk access for storing and retrieving data. They grouped incoming data into clusters and stored them instead of raw data.…”

“…Clustering streaming data is becoming important due to the availability of large amounts of data recorded from everyday transactions, web click streams, telephone records, web documents, sensors data, environmental data, and network monitoring [7], [13], [17], [20], [19], [21]. The sources of streaming data are increasing as technology for recording events is becoming cheaper.…”

An Adaptive Sweep-Circle Spatial Clustering Algorithm Based on Gestalt

“…,X i } according to the memory size. A load monitor [53] ensures that the loading of spatial data fits the main memory.…”

Data stream clustering

“…Synthetic data sets are usually preferred because testing hypotheses like noiserobustness, and scaling for high-dimensionality are easier to perform with synthetic data. Examples of artificially generated data sets are: (i) data generated by varying Gaussian distributions [Aggarwal et al 2003;Wan et al 2008;Dang et al 2009]; (ii) data generated by the IBM synthetic data generator [Ong et al 2004]; (iii) data simulating a taxi location tracking application [Cho et al 2006]; and (iv) data sets formed by arbitrarily-shaped clusters, like those presented in Figure 12.…”