A survey of parallel execution strategies for transitive closure and logic programs

Abstract

An important feature of database technology of the nineties is the use of parallelism for speeding up the execution of complex queries. This technology is being tested in several experimental database architectures and a few commercial systems for conventional select-project-join queries. In particular, hash-based fragmentation is used to distribute data to disks under the control of different processors in order to perform selections and joins in parallel. With the development of new query languages, and in particular with the definition of transitive closure queries and of more general logic programming queries, the new dimension of recursion has been added to query processing. Recursive queries are complex; at the same time, their regular structure is particularly suited for parallel execution, and parallelism may give a high efficiency gain. We survey the approaches to parallel execution of recursive queries that have been presented in the recent literature. We observe that research on parallel execution of recursive queries is separated into two distinct subareas, one focused on the transitive closure of Relational Algebra expressions, the other one focused on optimization of more general Datalog queries. Though the subareas seem radically different because of the approach and formalism used, they have many common features. This is not surprising, because most typical Datalog queries can be solved by means of the transitive closure of simple algebraic expressions. We first analyze the relationship between the transitive closure of expressions in Relational Algebra and Datalog programs. We then review sequential methods for evaluating transitive closure, distinguishing iterative and direct methods. We address the parallelization of these methods, by discussing various forms of parallelization. Data fragmentation plays an important role in obtaining parallel execution; we describe hash-based and semantic fragmentation. Finally, we consider Datalog queries, and present general methods for parallel rule execution; we recognize the similarities between these methods and the methods reviewed previously, when the former are applied to linear Datalog queries. We also provide a quantitative analysis that shows the impact of the initial data distribution on the performance of methods.

Query evaluation techniques for large databases

“…Recursive queries are omitted from this survey, because the entire field of recursive query processing -optimization rules and heuristics, selectivity and cost estimation, algorithms and their parallelization -is still developing rapidly; suffice it to point to two recent surveys [Bancilhon and Ramakrishnan 1986;Cacace, Ceri, and Houtsma 1993].…”

Parallel processing of graph reachability in databases

“…These works have also performed the comparison for a given t--the amount of time needed in order to generate a new tuple in the transitive closure, t depends on both the architecture and the input graph, and again, it is not clear how t can be estimated efficiently. As another example, Cacace, Ceri, and Houtsma (6) have analyzed the speedup of strategies for input graphs that have a regular topology (such as lists, grid, etc.). The analyzed strategies are designed for input graphs of arbitrary topology, but it is not clear how to estimate the speedup for such graphs.…”

Towards Scalable Non-Monotonic Stream Reasoning via Input Dependency Analysis

“…It means that they can contribute to infer a new fact by firing a single rule (condition (i)) or multiple rules (condition (ii)). The third one identifies a self-loop 5 of any predicate if its father node has a self-loop. The self-loop of a predicate describes the dependencies among ground atoms of that predicate.…”

“…The parallel strategies were important features of database technology in the nineties in order to speeding up the execution of complex queries [5]. In Semantic Web, the parallelism in reasoning has been studied in [17], [21], [19], [20] where a set of machines is assigned a partition of the parallel computation.…”