Flux: an adaptive partitioning operator for continuous query systems

Shah, Mehul A.; Hellerstein, Joseph M.; Chandrasekaran, S.; Franklin, Michael J.

doi:10.1109/icde.2003.1260779

Cited by 202 publications

(245 citation statements)

References 25 publications

Supporting

Mentioning

244

Contrasting

Unclassified

Order By: Relevance

“…The complete result is then the union of that produced by the original operator with that produced by the replacement operators; this union may not be carried out explicitly, as replacement operators may simply be planted within a suspended plan, which then resumes evaluation. The suspension and resumption of plans has been discussed in the literature (e.g., [14]). …”

Section: Replacing Operatorsmentioning

confidence: 99%

“…Since the value of symbol ([[ ]]) represents the result of a plan fragment, two or more results can be unioned, so (14) becomes:…”

Section: Replacing Operatorsmentioning

confidence: 99%

“…We cannot point to any cases in which published adaptive strategies have subsequently been shown to produce incorrect results, but contend that certain forms of adaptation may benefit from a more formal approach. We observe that adaptive strategies may be associated with complex protocols for halting, revising and resuming execution plans (e.g., [14]), and that certain categories of runtime change may only be fully explored when the safety net of a formal foundation is in place. One such category, which is the focus of this paper, is the replacement of operators in pipelined query plans.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Eurviriyanukul

Fernandes

Paton

2006

Current Trends in Database Technology – EDBT 2006

View full text Add to dashboard Cite

Abstract. Adaptive query processors make decisions as to the most effective evaluation strategy for a query based on feedback received while the query is being evaluated. In essence, any of the decisions made by the optimizer (e.g., on operator order or on which operators to use) may be revisited in an adaptive query processor. This paper focuses on changes to physical operators (e.g., the specific join operators used, such as hash-join or merge-join) in pipelined query evaluators. In so doing, the paper characterizes the runtime properties of pipelined operators in a way that makes explicit when specific operators may be replaced, and that allows the validity of operator replacements to be proved. This is illustrated with reference to the substitution of join operators during their evaluation.

show abstract

Section: Replacing Operatorsmentioning

confidence: 99%

“…Since the value of symbol ([[ ]]) represents the result of a plan fragment, two or more results can be unioned, so (14) becomes:…”

Section: Replacing Operatorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Eurviriyanukul

Fernandes

Paton

2006

Current Trends in Database Technology – EDBT 2006

View full text Add to dashboard Cite

show abstract

“…(2) Intermediate: the same as Elementary except foreign query fragments are given higher migration priori- ties than the native query fragments. In previous work, such as [9,14], data flow relationship is not considered. Hence their effects on the communication cost can be well represented by the Elementary algorithm.…”

Section: Load Selection Heuristicsmentioning

confidence: 99%

“…However, the authors did not propose any complete and practical strategies. In Flux [9], a dynamic load balancing strategy for the horizontal (or intra-operator) parallelism was employed. While the mechanism was developed in the context of continuous queries, a centralized synchronous controller is used to collect workload information and to make load balancing decisions.…”

Section: Related Workmentioning

confidence: 99%

Efficient Dynamic Operator Placement in a Locally Distributed Continuous Query System

Zhou

Ooi

Tan

et al. 2006

On the Move to Meaningful Internet Systems 2006: CoopIS, DOA, GADA, and ODBASE

View full text Add to dashboard Cite

Abstract. In a distributed processing environment, the static placement of query operators may result in unsatisfactory system performance due to unpredictable factors such as changes of servers' load, data arrival rates, etc. The problem is exacerbated for continuous (and long running) monitoring queries over data streams as any suboptimal placement will affect the system for a very long time. In this paper, we formalize and analyze the operator placement problem in the context of a locally distributed continuous query system. We also propose a solution, that is asynchronous and local, to dynamically manage the load across the system nodes. Essentially, during runtime, we migrate query operators/fragments from overloaded nodes to lightly loaded ones to achieve better performance. Heuristics are also proposed to maintain good data flow locality. Results of a performance study shows the effectiveness of our technique.

show abstract

Autonomics and data management

Paton

2008

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYTraditionally, database management systems have been associated with high-cost, high-quality functionalities. That is, powerful capabilities are provided, but only in response to careful design, procurement, deployment and administration. This has been successful in many contexts, but in an environment in which data are available in increasing quantities under the management of a growing collection of distributed applications, and where effective use of available data often provides a competitive edge, there is a requirement for various benefits of a comprehensive data management infrastructure to be made available with rather fewer of the costs. If this requirement is to be met, automation will need to be deployed much more widely and systematically in data management platforms. This paper reviews recent results on autonomic data management, makes a case that current practice presents significant opportunities for further development and argues that comprehensive support for automation should be central to future data management infrastructures, whether centralized or distributed.

show abstract

Flux: an adaptive partitioning operator for continuous query systems

Abstract: The

Cited by 202 publications

References 25 publications

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Efficient Dynamic Operator Placement in a Locally Distributed Continuous Query System

Autonomics and data management

Contact Info

Product

Resources

About