A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Abstract

Abstract. Adaptive query processors make decisions as to the most effective evaluation strategy for a query based on feedback received while the query is being evaluated. In essence, any of the decisions made by the optimizer (e.g., on operator order or on which operators to use) may be revisited in an adaptive query processor. This paper focuses on changes to physical operators (e.g., the specific join operators used, such as hash-join or merge-join) in pipelined query evaluators. In so doing, the paper characterizes the runtime properties of pipelined operators in a way that makes explicit when specific operators may be replaced, and that allows the validity of operator replacements to be proved. This is illustrated with reference to the substitution of join operators during their evaluation.

Time-completeness trade-offs in record linkage using adaptive query processing

“…As formally shown in [11], a sufficient condition for a "safe" switch, i.e., one where we can correctly compute the remainder of the join result after the switch without re-processing the tuples seen so far, is that the operators expose a particular state, called quiescent. In iterator-based evaluation [16], an operator implements three operations, i.e., OPEN(), NEXT() and CLOSE() (see Fig.…”

“…An actual execution trace is a path through the diagram. As shown in [11], some of the states N in the execution trace, i.e., the state the join algorithm is in when a call to NEXT() has concluded, are indeed quiescent states, making iterator-based algorithms a suitable choice for our purposes. The precise characterization of which N states are quiescent states is specific to each join algorithm.…”

“…This idea has proven viable for pipelined query plans [11], primarily as a dynamic optimization technique to improve the performance of a complex query, in cases where the initial query plan produced by the optimizer proves inefficient.…”

A Logical Framework for Monitoring and Evolving Software Components

“…Consider, for example, supervisory control systems for, say, reactive planning [12], or systems for adaptive querying (evaluating queries over changing databases [9]), or responsive memory management (variable capacity memory allocation), or data structure repair [8], or business process modelling (businesses which adapt their processes according to internal and external imperatives [1]), or hybrid systems [14] which change their computational behaviour in response to environmental factors which they may themselves influence [14]. On longer timescales, computational systems may evolve either through user action or automated updates e.g.…”

Automation Everywhere: Autonomics and Data Management

“…As the executing systems are complex, and the changes made to their behaviour may be non-trivial, it may be desirable to have certain guarantees as to the behaviour of a technique. Such guarantees could, for example, place bounds on the worst case performance that adaptation could lead to [19], or demonstrate that the outcome of a request is sure to be unchanged by an adaptation [12]. However, rather few proposals for adaptive techniques are accompanied by formal characterisations of their semantics or behavioural guarantees, and generic techniques for specifying and reasoning about such systems seem not to be well established [4].…”