Maximizing the Output Rate of Multi-Way Join Queries over Streaming Information Sources

Abstract

Recently there has been a growing interest in join query evaluation for scenarios in which inputs arrive at highly variable and unpredictable rates. In such scenarios, the focus shifts from completing the computation as soon as possible to producing a prefix of the output as soon as possible. To handle this shift in focus, most solutions to date rely upon some combination of streaming binary operators and "on-the-fly" execution plan reorganization. In contrast, we consider the alternative of extending existing symmetric binary join operators to handle more than two inputs. Toward this end, we have completed a prototype implementation of a multi-way join operator, which we term the "MJoin" operator, and explored its performance. Our results show that in many instances the MJoin produces outputs sooner than any tree of binary operators. Additionally, since MJoins are completely symmetric with respect to their inputs, they can reduce the need for expensive runtime plan reorganization. This suggests that supporting multi-way joins in a single, symmetric, streaming operator may be a useful addition to systems that support queries over input streams from remote sites.

Lifting the Burden of History from Adaptive Query Processing1

“…Another instance where such flexibility could be useful is when we are executing sliding window queries over streaming data [14,3,2,7,19]. Storing and reusing intermediate results is problematic in such a setting, because when a window on a base relation slides, some base-table …”

Processing Sliding Window Multi-Joins in Continuous Queries over Data Streams

“…We also know of no previous work on the join ordering problem in the context of sliding windows over data streams, although this problem is identified in the context of optimizing for the highest output rate of queries over infinite streams [28,29]. Generally, main-memory join ordering techniques focus on pushing expensive predicates to the top of the plan (see, e.g.…”

“…Viglas et al [29] have developed a multi-way version of the XJoin called the MJoin. Moreover, Viglas and Naughton [28] propose a rate-based query optimization model for continuous queries over data streams, which is relevant if the input rate changes with time, in which case the output rate of a join also changes with time.…”

iJoin: Importance-Aware Join Approximation over Data Streams

“…These schemes are orthogonal to our work and have been studied for approximating results when all tuples do not fit in memory. XJoin [29] and MJoin [30] produce exact results. In XJoin, some tuples are flushed to the disk when the data stream overwhelm the main memory.…”

“…When memory is full, [29][30][31]20] propose randomly evicting a tuple from the join-memory. [22] argues that this scheme is likely to produce sub-optimal results, and proposes various heuristics to maximize the output size.…”