Partial results in database systems

Abstract

As the size and complexity of analytic data processing systems continue to grow, the effort required to mitigate faults and performance skew has also risen. However, in some environments we have encountered, users prefer to continue query execution even in the presence of failures (e.g., the unavailability of certain data sources), and receive a "partial" answer to their query. We explore ways to characterize and classify these partial results, and describe an analytical framework that allows the system to perform coarse to fine-grained analysis to determine the semantics of a partial result. We propose that if the system is equipped with such a framework, in some cases it is better to return and explain partial results than to attempt to avoid them.

Estimating the Impact of Unknown Unknowns on Aggregate Query Results

“…Missing data is also well studied in databases [40,37,21]; however, as traditional RDBMS query processing function under the closed world assumption, they do not consider unknown unknowns as part of the query processing and largely consider it a data cleaning aspect.…”

“…Recent works [21,41] defined database completeness in a partly open world semantic (i.e., database can be incomplete, which causes incorrect query results) and use the completeness information to denote the completeness of query results. Similar in spirit to our work, they investigate the impact on query results of entire database records that may be missing [41]; however, they also assume the knowledge of population size (e.g., there are 7 days in a week, there are this many cities in France) to define the completely missing records and measure the completeness.…”

Correctness of SQL Queries on Databases with Nulls

“…One possibility is to adapt the approach of [24] that shows how to define certainty based on the semantics of inputs and outputs of queries. At the level of missing information, we would like to see whether our translations could help with deriving partial answers to SQL queries, when parts of a database are missing, as in [20].…”

Identifying the Extent of Completeness of Query Answers over Partially Complete Databases

“…is used in query answering. Examples are loosely coupled cloud databases such as AzureDB [16,15], crowd-sourced collaborative projects such as Wikipedia, and warehouses collecting monitoring data such as Darkstar [14].…”

“…To address incompleteness in these scenarios, many different mechanisms have been investigated. For cloud databases, Lang et al [16,15] identified the sources of incompleteness and traced how it is propagated through a query plan. For Wikipedia, a template has been introduced that allows lists to be labeled as complete.…”