Parallel-Correctness and Transferability for Conjunctive Queries

Abstract

A dominant cost for query evaluation in modern massively distributed systems is the number of communication rounds. For this reason, there is a growing interest in single-round multiway join algorithms where data is first reshuffled over many servers and then evaluated in a parallel but communication-free way. The reshuffling itself is specified as a distribution policy. We introduce a correctness condition, called parallel-correctness, for the evaluation of queries w.r.t. a distribution policy. We study the complexity of parallel-correctness for conjunctive queries as well as transferability of parallelcorrectness between queries. We also investigate the complexity of transferability for certain families of distribution policies, including, for instance, the Hypercube distribution.

Logical Aspects of Massively Parallel and Distributed Systems

“…Ameloot et al [14] present a framework for reasoning about the correctness of such one-round MPC algorithms for the evaluation of queries but under arbitrary distribution policies. To target the widest possible range of repartitioning strategies, the initial distribution phase is modeled by a distribution policy that can be any mapping from facts to subsets of servers.…”

“…Theorem 4.8. [14] Problems PC(CQ, P) and PCI(CQ, P) are Π p 2 -complete, for every policy class P ∈ {P fin } ∪ P npoly . We note that Proposition 4.6 continues to hold true in the presence of union and inequalities (under a suitable definition of minimal valuation for unions of CQs) leading to the same complexity bounds as stated in Theorem 4.8 [33].…”

“…In the context of multiple query evaluation, where an optimizer tries to automatically partition the base data across multiple nodes to achieve overall optimal performance for a specific workload (see, e.g., [46,51]), it makes sense to consider scenarios in which such reshuffling can be avoided. To this end, parallel-correctness transfer was introduced in [14] which states that a subsequent query Q can always be evaluated over a distribution for which a query Q is parallelcorrect.…”

“…When the defining condition of "covers" is spelled out by rewriting "minimal valuations" one gets a characterization 3 The terminology saturates, strongly saturates, and covers, was not used in [14] but was introduced in [15] and also used in [16]. with a Π3-structure.…”

“…[14] pc-trans(CQ) is Π p 3 -complete. We note that the same complexity bounds continue to hold in the presence of inequalities and for unions of conjunctive queries [15].…”