AFFINITY: Efficiently querying statistical measures on time-series data

Abstract

Abstract. Computing statistical measures for large databases of time series is a fundamental primitive for querying and mining time-series data [1][2][3][4][5][6]. This primitive is gaining importance with the increasing number and rapid growth of time series databases. In this paper we introduce a framework for efficient computation of statistical measures by exploiting the concept of affine relationships. Affine relationships can be used to infer statistical measures for time series from other related time series instead of directly computing them; thus, reducing the overall computation cost significantly. The resulting methods show at least one order of magnitude improvement over the best known methods. To the best of our knowledge, this is the first work that presents an unified approach for computing and querying several statistical measures on time-series data. Our approach includes three key components, which exploit affine relationships. First, the AFCLST algorithm that clusters the time-series data such that high-quality affine relationships could be easily found. Second, the SYMEX algorithm that uses the clustered time series and efficiently computes the desired affine relationships. Third, the SCAPE index structure that produces a many-fold improvement in the performance of processing several statistical queries by seamlessly indexing the affine relationships. Finally, we establish the effectiveness of our approaches by performing comprehensive experimental evaluation using real datasets.

Dominant Data Set Selection Algorithms for Electricity Consumption Time-Series Data Analysis Based on Affine Transformation

“…Therefore, the (ε, δ) principle is suitable for the macrodata analysis rather than the micro-data query. Although the dynamic statistics of partial TSD was proposed in [42] based on the affine transformation theory, it failed to represent the complete TSD information well. Unlike the existing methods, in this study, we implement the real-time data processing on the dominant dataset using the small-scale dominant dataset from large-scale TSD based on the given accuracy of information representation.…”

“…Inspired by the work in [42], we further introduce the terms: public object vector, central object vector, and sample object vector in this work, where it is assumed that • u represents a public object vector,…”

SigCO: Mining significant correlations via a distributed real-time computation engine

“…Various indexing techniques for querying the correlations of static time-series data stored in a centralized system have been proposed in [11], [12], [20], [23]. Such techniques are not suitable for our dynamic environment, where the index maintenance cost incurs high processing latency.…”

“…Even though dimensionality reduction methods are not the focus of this paper, we briefly discuss how our framework can incorporate such techniques [12], [20]. Orthonormal transformation based dimensionality reduction (e.g., discrete Fourier transformation (DFT), random projections, etc.)…”

“…Due to the explosion of devices producing time-series data (e.g., sensor networks, mobile phones, Internet of Things) [5], [8], [20], for contemporary large-scale time series mining applications, it is not feasible to simply load real-time time series data into a traditional stream processing system [4], [5] run on a standalone machine, which cannot handle the rapidly increasing amount of time series data. This has led to the development of many distributed, fault-tolerant, and realtime computation systems [2], [3], [13], [24].…”