An Efficient Filter Strategy for Theta-Join Query in Distributed Environment

Liu, Wenjie; Li, Zhanhuai; Zhou, Yuntao

doi:10.1109/icppw.2017.24

Cited by 4 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, they should be reduced as much as possible. As a result, numerous efforts have been made by both industry and academia from different aspects to optimise the efficiency of the theta-join operation 32,33,34,35 .…”

Section: Figurementioning

confidence: 99%

“…In this paper, we propose a novel fast online theta-join algorithm, called Prefap. Instead of performing filtering only after partitioning as in 32,33,34,35 , it makes sense to perform a pre-filtering based on the theta condition before the partitioning takes place as it can not only reduce the amount of data involved in the partitioning but also make the partitioning more fine-grained. Since our partitioning is based on the range boundaries calculated using the minimum and the maximum values of the data streams, the pre-filtering of the data streams is possible to condense the range between the minimum and the maximum value, hence making the partition more fine-grained.…”

Section: Figurementioning

confidence: 99%

“…Moreover, this pre-filtering mechanism does not suffer from the severe overhead caused by the time-consuming sorting operation 36,37 . As opposed to 32,33,34 , the proposed algorithm requires neither inter-partition nor intra-partition ordering.…”

Section: Figurementioning

confidence: 99%

“…With this mechanism, the data stream will be partitioned based on the amalgamated partitioning boundary that fuses the partitioning information of data streams. Hence, worthless Cartesian products between partitions that are deemed impossible to possess valid theta-join results will be avoided, and hence making the algorithm more efficient, which is an improvement to 32,33,34,35 .…”

Section: Figurementioning

confidence: 99%

“…As a result, numerous efforts have been made by both industry and academia from different aspects to optimize the efficiency of the theta-join operation. [32][33][34][35] F I G U R E 1 An illustrative example of a theta-join predicate  ⋈  (.B > .B), its input and its corresponding output. In this example, there are only four phone-laptop combinations that satisfy the price of the phone is more expensive than laptop…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Toward fast theta‐join: A prefiltering and amalgamated partitioning approach

Wang

Fan

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

As one of the most useful online processing techniques, the theta‐join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed environment, which is typically characterized by reducing the number of Cartesian products as much as possible. In this article, we design and implement a novel fast theta‐join algorithm, called Prefap, by developing two distinct techniques—prefiltering and amalgamated partitioning—based on the state‐of‐the‐art FastThetaJoin algorithm to optimize the efficiency of the theta‐join operation. Firstly, we develop a prefiltering strategy before data streams are partitioned to reduce the amount of data to be involved and benefit a more fine‐grained partitioning. Secondly, to avoid the data streams being partitioned in a coarse‐grained isolated manner and improve the quality of the partition‐level filtering, we introduce an amalgamated partitioning mechanism that can amalgamate the partitioning boundaries of two data streams to assist a fine‐grained partitioning. With the integration of these two techniques into the existing FastThetaJoin algorithm, we design and implement a new framework to achieve a decreased number of Cartesian products and a higher theta‐join efficiency. By comparing with existing algorithms, FastThetaJoin in particular, we evaluate the performance of Prefap on both synthetic and real data streams from two‐way to multiway theta‐join to demonstrate its superiority.

show abstract

Section: Figurementioning

confidence: 99%

Section: Figurementioning

confidence: 99%

Section: Figurementioning

confidence: 99%

Section: Figurementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations