Conjunctive selection conditions in main memory

Abstract

We consider the fundamental operation of applying a conjunction of selection conditions to a set of records. With large main memories available cheaply, systems may choose to keep the data entirely in main memory, in order to improve query and/or update performance.The design of a data-intensive algorithm in main memory needs to take into account the architectural characteristics of modern processors, just as a disk-based method needs to consider the physical characteristics of disk devices. An important architectural feature that influences the performance of main memory algorithms is the branch misprediction penalty. We demonstrate that branch misprediction has a substantial impact on the performance of an algorithm for applying selection conditions. We describe a space of "query plans" that are logically equivalent, but differ in terms of performance due to variations in their branch prediction behavior. We propose a cost model that takes branch prediction into account, and develop a query optimization algorithm that chooses a plan with optimal estimated cost. We also develop an efficient heuristic optimization algorithm.We provide experimental results for a case study based on an event notification system. Our results show the effectiveness of the proposed optimization techniques. Our results also demonstrate that significant improvements in performance can be obtained by applying a methodology that takes branch misprediction latency into account.

Efficiently compiling efficient query plans for modern hardware

“…One important line of work is reducing the impact of branching, where [14] showed how to combine conjunctive predicates such that the trade-off between number of branches and number of evaluated predicates is optimal. Other work has looked at processing individual expressions more efficiently by using SIMD instructions [12,15].…”

Predictable performance for unpredictable workloads

“…//get probability p of a according to Z Z[a] ← 0 ; //remove a from Z Z ← Z/ (1 − p) ; //re-normalize remaining Z P ← P ∪ a ; //add a to result set P timizations described by Boncz et al [4,5] and Ross [20,21].…”

“…The reason is that the cost of scanbased on-line query processing is dominated by predicate evaluation. Optimizing the evaluation of single queries has previously been studied by Ross [20,21]. In this paper, we extended the idea to sets of queries, and introduced cacheconscious query/update-data join algorithms based on predicate indexing: Index Union Join and Index Union Update Join.…”

A Query Index for Stream Data Using Interval Skip Lists Exploiting Locality

“…For example, when the data was 5.5, QSet was {q1,q2} and a new data item 13 arrives, [4, N ull) represented by TIL [3] contains 13. Therefore, the procedure removes the search overhead starting from Header.…”

“…In this case, if all registered queries are invoked whenever a stream data item arrives, the system performance degrades. Therefore, Query indexes are built on registered continuous queries [3]. Upon each stream data arrives, a CQ engine searches for matching queries using these indexes.…”