Query-driven parallel exploration of large datasets

Atanasov, Atanas; Srinivasan, Madhusudhanan; Weinzierl, Tobias

doi:10.1109/ldav.2012.6378972

Cited by 4 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In modern big data analytics applications, supported by big data infrastructures, predictive analytics [2], [3] and exploratory analysis are commonly based on statistical aggregation operators over the results of 'complex' queries [5]. Such complex, aggregate queries typically involve large datasets (which may themselves be the result of linking of other different datasets) and a number of range predicates over multidimensional vectors, structured, semi-and unstructured data.…”

Section: Introductionmentioning

confidence: 99%

Learning to accurately COUNT with query-driven predictive analytics

Anagnostopoulos

Triantafillou

2015

2015 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Abstract-We study a novel solution to executing aggregation (and specifically COUNT) queries over large-scale data. The proposed solution is generally applicable, in the sense that it can be deployed in environments in which data owners may or may not restrict access to their data and allow only 'aggregation operators' to be executed over their data. For this, it is based on predictive analytics, driven by queries and their results. We propose a machine learning (ML) framework for the task (which can be adapted for different aggregates as well). We focus on the widely used set-cardinality (i.e., COUNT) aggregation operator, as it is a fundamental operator for both internal data system optimisations and for aggregation-query analytics. We contribute a novel, query-driven ML model whose goals are to: (i) learn the query space (access patterns), (ii) associate (complex) aggregation queries with the cardinality of their results, (iii) define query similarity and use it to predict the cardinality of the answer set of an ad-hoc incoming query. Our ML model incorporates incremental learning algorithms for ensuring high prediction accuracy even when both the querying patterns and the underlying data change. The significance of contribution lies in that it (i) is the only query-driven solution applicable over general environments which include restrictedaccess data, (ii) offers incremental learning adjusted for arriving ad-hoc queries, which is well suited for big data analytics, and (iii) offers a performance (in terms of prediction accuracy and time, and memory requirements) that is superior to datacentric approaches. We provide a comprehensive performance evaluation of our model, evaluating its sensitivity and comparative advantages versus acclaimed data-centric methods (self-tuning histograms, sampling, and multidimensional histograms).

show abstract

Section: Introductionmentioning

confidence: 99%

Learning to accurately COUNT with query-driven predictive analytics

Anagnostopoulos

Triantafillou

2015

2015 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

show abstract

“…In predictive analytics [Lin et al 2014], data exploration is commonly based on aggregation operators (e.g., count, average, median) over the results of analytics queries. Such queries typically involve datasets (which may themselves be the result of linking of other different datasets) and a number of selection predicates (e.g., range predicates over multi-dimensional vectors), which are important for predictive analytics and exploratory analysis [Atanasov et al 2012], [Balac et al 2013]. As (i) query-driven data exploration is becoming increasingly important in the presence of largescale data [Gosink et al 2011], and (ii) answering (aggregations over) analytics queries is a fundamental exploration task [Chaudhuri et al 2014], it is important to study how to process them in modern scale-out data systems.…”

Section: Introductionmentioning

confidence: 99%

Query-Driven Learning for Predictive Analytics of Data Subspace Cardinality

Anagnostopoulos

Triantafillou

2017

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

Fundamental to many predictive analytics tasks is the ability to estimate the cardinality (number of vectors) of multidimensional data subspaces, defined by query selections over datasets. This is crucial for data analysts dealing with e.g., interactive data subspace explorations, data subspace visualizations, and in query processing optimization. However, in many modern data systems, predictive analytics may be (i) too costly money-wise, e.g., in clouds, (ii) unreliable, e.g., in modern Big Data query engines, where accurate statistics are difficult to obtain/maintain, or (iii) infeasible, e.g., for privacy issues. We contribute a novel, querydriven, function estimation model of analyst-defined data subspace cardinality. The proposed estimation model is highly accurate in terms of prediction and accommodating the well known selection queries: multidimensional range and distance-nearest neighbors (radius) queries. Our function estimation model: (i) quantizes the vectorial query space, by learning the analysts' access patterns over a data space, (ii) associates query vectors with their corresponding cardinalities of the analyst-defined data subspaces, (iii) abstracts and employs query vectorial similarity to predict the cardinality of an unseen/unexplored data subspace, and (iv) identifies and adapts to possible changes of the query sub-spaces based on the theory of optimal stopping. The proposed model is decentralized, facilitating the scaling-out of such predictive analytics queries. The research significance of the model lies in that (i) it is an attractive solution when data-driven statistical techniques are undesirable or infeasible, (ii) it offers a scale-out, decentralized training solution, (iii) it is applicable to different selection query types, and (iv) it offers a performance that is superior to that of certain data-driven approaches.

show abstract