Collecting and Maintaining Just-in-Time Statistics

El-Helw, Amr; Ilyas, Ihab F.; Lau, Wing Cheong; Markl, Volker; Zuzarte, Calisto

doi:10.1109/icde.2007.367897

Cited by 15 publications

(10 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore it is critical that the techniques used for identifying importance of expressions are lightweight and simple to implement. We note that the MNSA technique proposed in [10] is too heavy-weight for our purpose, and the sensitivity analysis techniques discussed in [15] is focused only on single-table expressions. Therefore, we focus on low overhead but intuitive measures for ranking expressions by importance.…”

Section: Identifying Important Expressionsmentioning

confidence: 99%

“…Our mechanisms can also apply for K-FK joins (Section 3.2). It is interesting to examine if the sensitivity analysis techniques presented in [15] can be extended for the case of joins.…”

Section: Related Workmentioning

confidence: 99%

“…In contrast, our pay-as-you-go approach incurs a bounded overhead on each query execution. The idea of Query-Specific Statistics (JITS) presented in [15] is focused on obtaining statistics of relevant single-table expressions. Our mechanisms can also apply for K-FK joins (Section 3.2).…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A pay-as-you-go framework for query execution feedback

2008

View full text Add to dashboard Cite

Past work has suggested that query execution feedback can be useful in improving the quality of plans by correcting cardinality estimation errors in the query optimizer. The state-of-the-art approach for obtaining execution feedback is "passive" monitoring which records the cardinality of each operator in the execution plan. We observe that there are many cases where even after repeated executions of the same query with use of feedback from passive monitoring, suboptimal choices in the execution plan cannot be corrected. We present a novel "pay-as-you-go" framework in which a query potentially incurs a small overhead on each execution but obtains cardinality information that is not available with passive monitoring alone. Such a framework can significantly extend the reach of query execution feedback in obtaining better plans. We have implemented our techniques in Microsoft SQL Server, and our evaluation on real world and synthetic queries suggests that plan quality can improve significantly compared to passive monitoring even at low overheads.

show abstract

Section: Identifying Important Expressionsmentioning

confidence: 99%

“…Our mechanisms can also apply for K-FK joins (Section 3.2). It is interesting to examine if the sensitivity analysis techniques presented in [15] can be extended for the case of joins.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A pay-as-you-go framework for query execution feedback

2008

View full text Add to dashboard Cite

show abstract

“…Several techniques have been also proposed in the literature that aim to deal with the problems that the inaccurate statistics cause during database query optimization (e.g., [28][29][30]). For example, to derive a subquery's cardinality prior to query execution, a query optimizer must take several assumptions regarding the distribution of the base data (e.g., data uniformity assumption) and the query predicates (e.g., predicate independence assumption).…”

Section: Related Workmentioning

confidence: 99%

Incorporating change detection in the monitoring phase of adaptive query processing

Tsamoura

Gounaris

Manolopoulos

2016

J Internet Serv Appl

View full text Add to dashboard Cite

Recent Big Data research typically emphasizes on the need to address the challenges stemming from the volume, velocity, variety and veracity aspects. However, another cross-cutting property of Big Data is volatility. In database technology, volatility is addressed with the help of adaptive query processing (AQP), which has become the dominant paradigm for executing queries in dynamic and/or streaming environments. As the characteristics of the runtime environment may vary significantly along time, AQP techniques employ a three-phase adaptivity loop to process the input queries, comprising feedback collection, analysis and re-optimization. In the monitoring phase, the standard approach is to collect feedback in a fixed-size sliding window. However, several problems arise when the techniques adopt a fixed-size sliding window for maintaining runtime collected feedback. In this work, we tackle this limitation and we propose a novel monitoring phase, which assesses the collected feedback rendering an AQP technique capable of taking more informed decisions during the subsequent phases. The proposed approach is non-intrusive to the state-of-the-art adaptivity loop and can adopt any state-of-the-art online change detection algorithm through its plug-and-play abstraction. Another contribution of this work is a novel algorithm for detecting changes in a filter's drop probability, called β-CUSUM. The potential of the novel monitoring phase and of β-CUSUM is experimentally evaluated using both real-world and synthetic datasets.

show abstract

“…Proof Sketch: -We show a reduction from the Set Cover problem [10]. The Set Cover problem takes as input a set U and a set of subsets of U, S = {S i }.…”

Section: Claimmentioning

confidence: 99%

Exact cardinality query optimization for optimizer testing

Chaudhuri¹,

Narasayya²,

Ramamurthy³

2009

Proc. VLDB Endow.

View full text Add to dashboard Cite

The accuracy of cardinality estimates is crucial for obtaining a good query execution plan. Today"s optimizers make several simplifying assumptions during cardinality estimation that can lead to large errors and hence poor plans. In a scenario such as query optimizer testing it is very desirable to obtain the "best" plan, i.e., the plan produced when the cardinality of each relevant expression is exact. Such a plan serves as a baseline against which plans produced by using the existing cardinality estimation module in the query optimizer can be compared. However, obtaining all exact cardinalities by executing appropriate subexpressions can be prohibitively expensive. In this paper, we present a set of techniques that makes exact cardinality query optimization a viable option for a significantly larger set of queries than previously possible. We have implemented this functionality in Microsoft SQL Server and we present results using the TPC-H benchmark queries that demonstrate their effectiveness.

show abstract

Collecting and Maintaining Just-in-Time Statistics

Cited by 15 publications

References 13 publications

A pay-as-you-go framework for query execution feedback

A pay-as-you-go framework for query execution feedback

Incorporating change detection in the monitoring phase of adaptive query processing

Exact cardinality query optimization for optimizer testing

Contact Info

Product

Resources

About