A magic approach to optimizing incremental relational expressions

Abstract

This paper is concerned with a transformation-based approach to update propagation in an extended version of Codd's relational algebra which allows for defining derived relations (even recursively). It is shown that the desired optimization effects of update propagation may be lost if no generalized selection pushing strategy is employed to the transformed algebra expressions. A possible solution is the application of the Magic Sets rewriting but this may lead to unstratifiability of the incremental expressions. For the efficient evaluation of Magic Sets transformed algebra expressions we propose to use the soft stratification approach because of the simplicity and efficiency of this technique.

Optimized incremental ETL jobs for maintaining data warehouses

“…In [4] we proposed to use Magic Sets for optimizing the state evaluation of incremental RA expressions. To this end, the tuples of delta relations are considered as abstract subqueries on the state relations occurring within the same relational expression.…”

“…In the Magic Updates (MU) approach [4], all state relations are optimized in this way. To this end, all relevant subqueries in the set of incremental RA rules ϕ(R) are collected first.…”

“…On the other hand, our method can also be applied to recursive ETL jobs (in case stratifiable recursion will be provided in the future) as MS is especially suited for optimizing recursive expressions. This time, however, the stratification property may even be lost but an efficient evaluation technique for such MU transformed expression has been already proposed in [4].…”

“…In contrast to the paper at hand, however, this work was focused on update propagation in the presence of incomplete information on source changes. The application of Magic Sets for an efficient computation of true updates has been already proposed by the authors in [4] but with no focus on the warehouse context.…”