Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches

Cormode, Graham; Garofalakis, Minos; Haas, Peter J.; Jermaine, Chris

doi:10.1561/1900000004

Cited by 311 publications

(89 citation statements)

References 237 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing work on approximate query answering in relational databases considers the use of synopses (histograms, join synopses, and wavelets) to speedup the evaluation of aggregate queries with the goal of quickly reporting the leading digits of the answers. A survey of these synopsis-based techniques has been recently compiled [8]. Their focus is not on deriving optimal bounds within a given language.…”

Section: Related Workmentioning

confidence: 99%

Anytime approximation in probabilistic databases

2013

View full text Add to dashboard Cite

This article describes an approximation algorithm for computing the probability of propositional formulas over discrete random variables. It incrementally refines lower and upper bounds on the probability of the formulas until the desired absolute or relative error guarantee is reached. This algorithm is used by the SPROUT query engine to approximate the probabilities of results to relational algebra queries on expressive probabilistic databases.

show abstract

Section: Related Workmentioning

confidence: 99%

Anytime approximation in probabilistic databases

2013

View full text Add to dashboard Cite

show abstract

“…Such dependencies are often discovered using sampling [76] or other summarization techniques [31]. Their approximate nature is often sufficient for certain tasks, and approximate dependencies can be used as input to the more rigorous task of detecting true dependencies.…”

Section: Conditional Partial and Approximate Solutionsmentioning

confidence: 99%

“…Sketches have been widely applied to answering approximate queries, data stream processing and estimating join sizes [37,54,111]. Cormode et al [31] give an overview of sketching and sampling for approximate query processing.…”

Section: Summaries and Sketchesmentioning

confidence: 99%

Profiling relational data: a survey

Abedjan¹,

2015

View full text Add to dashboard Cite

Profiling data to determine metadata about a given dataset is an important and frequent activity of any IT professional and researcher and is necessary for various use-cases. It encompasses a vast array of methods to examine datasets and produce metadata. Among the simpler results are statistics, such as the number of null values and distinct values in a column, its data type, or the most frequent patterns of its data values. Metadata that are more difficult to compute involve multiple columns, namely correlations, unique column combinations, functional dependencies, and inclusion dependencies. Further techniques detect conditional properties of the dataset at hand. This survey provides a classification of data profiling tasks and comprehensively reviews the state of the art for each class. In addition, we review data profiling tools and systems from research and industry. We conclude with an outlook on the future of data profiling beyond traditional profiling tasks and beyond relational databases.

show abstract

“…(Cormode et al 2012). These two types of data synopses -samples and summaries -are to some extent combinable (Ramnarayan et al 2016).…”

Section: Literature Overviewmentioning

confidence: 99%

“…This may turn out to be even more important than the ability to store and process the original data, especially given the fact that perfect data access is questionable in many environments (Narayanan et al 2008). For the knowledge representation purposes, we utilize enhanced histogram structures (Cormode et al 2012). For each original data pack, its histogram contains information about dynamically derived range-based bars and special values that differ from neighboring values of the given column by means of their frequencies in the given packrow.…”

Section: Histograms Special Values and Domain Representationsmentioning

confidence: 99%

A new approximate query engine based on intelligent capture and fast transformations of granulated data summaries

Ślęzak

Glick

Betliński³

et al. 2017

J Intell Inf Syst

View full text Add to dashboard Cite

We outline the processes of intelligent creation and utilization of granulated data summaries in the engine aimed at fast approximate execution of analytical SQL statements. We discuss how to use the introduced engine for the purposes of ad-hoc data exploration over large and quickly increasing data collected in a heterogeneous or distributed fashion. We focus on mechanisms that transform input data summaries into result sets representing query outcomes. We also illustrate how our computational principles can be put together with other paradigms of scaling and harnessing data analytics.

show abstract

Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches

Cited by 311 publications

References 237 publications

Anytime approximation in probabilistic databases

Anytime approximation in probabilistic databases

Profiling relational data: a survey

A new approximate query engine based on intelligent capture and fast transformations of granulated data summaries

Contact Info

Product

Resources

About