Query-Driven Learning for Predictive Analytics of Data Subspace Cardinality

Abstract

Fundamental to many predictive analytics tasks is the ability to estimate the cardinality (number of vectors) of multidimensional data subspaces, defined by query selections over datasets. This is crucial for data analysts dealing with e.g., interactive data subspace explorations, data subspace visualizations, and in query processing optimization. However, in many modern data systems, predictive analytics may be (i) too costly money-wise, e.g., in clouds, (ii) unreliable, e.g., in modern Big Data query engines, where accurate statistics are difficult to obtain/maintain, or (iii) infeasible, e.g., for privacy issues. We contribute a novel, querydriven, function estimation model of analyst-defined data subspace cardinality. The proposed estimation model is highly accurate in terms of prediction and accommodating the well known selection queries: multidimensional range and distance-nearest neighbors (radius) queries. Our function estimation model: (i) quantizes the vectorial query space, by learning the analysts' access patterns over a data space, (ii) associates query vectors with their corresponding cardinalities of the analyst-defined data subspaces, (iii) abstracts and employs query vectorial similarity to predict the cardinality of an unseen/unexplored data subspace, and (iv) identifies and adapts to possible changes of the query sub-spaces based on the theory of optimal stopping. The proposed model is decentralized, facilitating the scaling-out of such predictive analytics queries. The research significance of the model lies in that (i) it is an attractive solution when data-driven statistical techniques are undesirable or infeasible, (ii) it offers a scale-out, decentralized training solution, (iii) it is applicable to different selection query types, and (iv) it offers a performance that is superior to that of certain data-driven approaches.

Scalable aggregation predictive analytics

“…Platforms such as MapReduce [14], Yarn [29], Spark [32] and Mahout [22] are nowadays commonplace. Predictive modeling [26], [23] and exploratory analysis [2,3,6,20] are commonly based on statistical aggregation operators over the results of exploration queries [4,7]. Such queries involve large datasets (which may themselves be the result of linking of other different datasets) and a number of range predicates over multidimensional data vectorial representation, structured, semi-and unstructured data.…”

Predictive intelligence to the edge: impact on edge analytics

“…This has been realized in many previous studies (McConnell and Skillicorn 2005;Tulone and Madden 2006;Goel and Imielinski 2001;Anagnostopoulos and Triantafillou 2014, 2015, 2017a. In this case, analytics tasks are carried out by the back-end system on the cloud only, and not by the SANs or ENs at the edge of the network, despite their increasing computing capacity.…”

Adaptive learning of aggregate analytics under dynamic workloads

“…Our work is related to prior work in analytical-query processing and in applied ML research communities and to prior work focusing on the benefits of the query-driven approach in analytical query processing and tuning [5,6,18,24]. Analytical queries nowadays are executed over underlying systems that provide either exact answers [21,26] or approximate answers [4,14,16,22,23] working over large big data clusters in DCs/CS requiring several orders of magnitude longer query response times.…”

“…Query-driven models are largely being deployed for both aggregate estimation [5,6] and for hyper-tuning [28] database systems. Unlike [5,6] our focus is on a wide variety of aggregate operators and not just COUNT for selectivity estimation. Furthermore, we address the crucial problem of detecting query pattern changes and adapting to them, which (to our knowledge) has not been addressed in this context before.…”