Optimal algorithms for ranked enumeration of answers to full conjunctive queries

Abstract

We study ranked enumeration of join-query results according to very general orders defined by selective dioids. Our main contribution is a framework for ranked enumeration over a class of dynamic programming problems that generalizes seemingly different problems that had been studied in isolation. To this end, we extend classic algorithms that find the k -shortest paths in a weighted graph. For full conjunctive queries, including cyclic ones, our approach is optimal in terms of the time to return the top result and the delay between results. These optimality properties are derived for the widely used notion of data complexity, which treats query size as a constant. By performing a careful cost analysis, we are able to uncover a previously unknown tradeoff between two incomparable enumeration approaches: one has lower complexity when the number of returned results is small, the other when the number is very large. We theoretically and empirically demonstrate the superiority of our techniques over batch algorithms, which produce the full result and then sort it. Our technique is not only faster for returning the first few results, but on some inputs beats the batch algorithm even when all results are produced.

Ranked Enumeration of Join Queries with Projections

“…Together, the above lemmas establish Theorem 1. We defer the full proofs to Appendix C. We also show how we can recover logarithmic delay guarantee for full queries from [26,65].…”

“…After materializing the results of the subquery induced by each node in the decomposition, the residual query becomes acyclic. Hence, we can apply our algorithm for acyclic queries directly obtaining the following: We now go one step further and extend our algorithm to queries that are unions of join-project queries (UCQs) using an idea introduced by [26,65]. A UCQ query is of the form 𝑄 = 𝑄 1 ∪𝑄 2 ∪• • •∪𝑄 𝑚 , where each 𝑄 𝑖 is a join-project query defined over the same projection attributes A. Semantically, 𝑄 (𝐷) = 𝑖 𝑄 𝑖 (𝐷).…”

“…We also compare the performance of our algorithm to other systems for cyclic queries. We choose four cyclic queries found commonly in practice inspired by [65]: four cycle, six cycle, eight cycle and bowtie query (two four cycles joined at a common attribute). Figure 10 shows the performance of our algorithm on the DBLP dataset for the sum function.…”

“…Efficient evaluation of join queries in the presence of ranking functions has been a subject of intense research in the database community. Recent work [18,26,65,67,69] has made significant progress in identifying optimal algorithms for enumerating query results in ranked order. In each of these works, the key idea is to perform on-the-fly sorting of the output via the use of priority queues by taking into account the query structure.…”

“…[18] considered the problem of top-k tree matching in graphs and proposed optimal algorithms by combining Lawler's procedure [44] with the ranking function. [65] introduced multiple dynamic programming algorithms that lazily populate the priority queues. [69] took a different approach where all possible candidates were eagerly inserted into the priority queues and [26] generalized these ideas to present a unified theory of ranked enumeration for full join queries.…”