Turbo-charging estimate convergence in DBO

Abstract

DBO is a database system that utilizes randomized algorithms to give statistically meaningful estimates for the final answer to a multi-table, disk-based query from start to finish during query execution. However, DBO's "time 'til utility" (or "TTU"; that is, the time until DBO can give a useful estimate) can be overly large, particularly in the case that many database tables are joined in a query, or in the case that a join query includes a very selective predicate on one or more of the tables, or when the data are skewed. In this paper, we describe Turbo DBO, which is a prototype database system that can answer multi-table join queries in a scalable fashion, just like DBO. However, Turbo DBO often has a much lower TTU than DBO. The key innovation of Turbo DBO is that it makes use of novel algorithms that look for and remember "partial match" tuples in a randomized fashion. These are tuples that satisfy some of the boolean predicates associated with the query, and can possibly be grown into tuples that actually contribute to the final query result at a later time.

Database Learning

“…This is because Verdict reaches a target error bound much earlier by combining its model with the raw answer of the AQP engine. [6,19,24,36,45,66,87]: Instead of continuously refining approximate answers and reporting them to the user, these engines simply take a time-bound from the user, and then they predict the largest sample size that they can process within the requested time-bound; thus, they minimize error bounds within the allotted time. For these engines, Verdict simply replaces the user's original time bound t 1 with a slightly smaller value t 1 − ǫ before passing it down to the AQP engine, where ǫ is the time needed by Verdict for inferring the improved answer and improved error.…”

Speculative Approximations for Terascale Distributed Gradient Descent Optimization

“…The database online aggregation literature has its origins in the seminal paper by Hellerstein et al [21]. We can broadly categorize this body of work into system design [30,7,13,2], online join algorithms [20,8,34], and methods to derive confidence bounds [19]. All of this work is targeted at single-node centralized environments.…”

Knowing when you're wrong

“…In particular, sampling has served as one of the most common and generic approaches to approximation of analytical queries [7,8,13,14,15,21,23,30,31]. The simplest form of sampling is simple random sampling with plug-in estimation.…”

“…Nearly three decades ago, Olken and Rotem [27] introduced random sampling in relational databases as a means to return approximate answers and reduce query response times. A large body of work has subsequently proposed different sampling techniques [7,8,13,14,15,21,23,30,31,36]. All of this Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

“…Unfortunately, deriving closed forms is often a manual, analytical process. As a result, closed-formbased S-AQP systems [7,8,13,14,15,21,31] are restricted to very simple SQL queries (often with only a single layer of basic aggregates like AVG, SUM, COUNT, VARIANCE and STDEV with projections, filters, and a GROUP BY). This has motivated the use of resampling methods like the bootstrap [23,30], since these methods require no such detailed analysis and can be applied to arbitrarily complex SQL queries.…”