Strategies for using additional resources in parallel hash-based join algorithms

Zhang, Xi; Kurç, Tahsin; Pan, Tai-Yu; Çatalyürek, Ümit V.; Narayanan, S.; Wyckoff, P.; Saltz, Joel

doi:10.1109/hpdc.2004.1323471

Cited by 8 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, we believe that the evaluation conducted in this work and the described results are of value to the community as a basis for understanding the merits of the approach. In the meantime, based on the advantages of our new implementation, we expect that our approach will also be efficient and easy to implement in more complex enviroments such as heterogeneous environments [43], which require more efficient strategies on resource management [28], [44].…”

Section: Related Workmentioning

confidence: 99%

Design and evaluation of small–large outer joins in cloud computing environments

Cheng

Tachmazidis

Kotoulas

et al. 2017

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Design and evaluation of small–large outer joins in cloud computing environments

Cheng

Tachmazidis

Kotoulas

et al. 2017

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

“…First, the spatial-temporal joins in MOD can not be supported by existing parallel join methods. In parallel, relational databases, joins are often conducted based on divide-and-conquer techniques, such as ABJ+ [30], CMD-Join [34], Grace Hash Join [32], SBABJ+ [37], EHJA [50], DER [46] and BSP [9]. However, spatial-temporal joins in MOD often make the data set un-dividable.…”

Section: Related Workmentioning

confidence: 99%

Enabling Smart Transportation Systems: A Parallel Spatio-Temporal Database Approach

Ding

Yang

Chi

et al. 2016

IEEE Trans. Comput.

View full text Add to dashboard Cite

We are witnessing increasing interests in developing "smart cities" which helps improve the efficiency, reliability, and security of a traditional city. An important aspect of developing smart cities is to enable "smart transportation," which improves the efficiency, safety, and environmental sustainability of city transportation means. Meanwhile, the increasing use of GPS devices has led to the emergence of big trajectory data that consists of large amounts of historical trajectories and real-time GPS data streams that reflect how the transportation networks are used or being used by moving objects, e.g., vehicles, cyclists, and pedestrians. Such big trajectory data provides a solid data foundation for developing various smart transportation applications, such as congestion avoidance, reducing greenhouse gas emissions, and effective traffic accident response, etc. Instead of proposing yet another specific smart transportation application, we propose the Parallel-Distributed Network-constrained Moving Objects Database (PD-NMOD), a general framework that manages big trajectory data in a scalable manner, which provides an infrastructure that is able to support a wide variety of smart transportation applications and thus benefiting the smart city vision as a whole. The PD-NMOD manages both transportation networks and trajectories in a distributed manner. In addition, the PD-NMOD is designed to support general SQL queries over moving objects and to efficiently process the SQL queries on big trajectory data in parallel. Such design facilitates smart transportation applications to retrieve relevant trajectory data and to conduct statistical analyses. Empirical studies on a large trajectory data set collected from 3,500 taxis in Beijing offer insight into the design properties of the PD-NMOD and offer evidence that the PD-NMOD is efficient and scalable.

show abstract

“…Various techniques have been proposed for distributed inner joins to handle skew [7,[13][14][15]. Often, the assumption is that inner join techniques can be simply applied to outer joins, as identified in [4].…”

Section: Related Work On Joinsmentioning

confidence: 99%

Robust and Efficient Large-Large Table Outer Joins on Distributed Infrastructures

Cheng

Kotoulas

Ward

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. Abstract. Outer joins are ubiquitous in many workloads but are sensitive to loadbalancing problems. Current approaches mitigate such problems caused by data skew by using (partial) replication. However, contemporary replication-based approaches (1) introduce overhead, since they usually result in redundant data movement, (2) are sensitive to parameter tuning and value of data skew and (3) typically require that one side is small. In this paper, we propose a novel parallel algorithm, Redistribution and Efficient Query with Counters (REQC), aimed at robustness in terms of size of join sides, variation in skew and parameter tuning. Experimental results demonstrate that our algorithm is faster, more robust and less demanding in terms of network bandwidth, compared to the state-of-the-art.

show abstract

Strategies for using additional resources in parallel hash-based join algorithms

Cited by 8 publications

References 19 publications

Design and evaluation of small–large outer joins in cloud computing environments

Design and evaluation of small–large outer joins in cloud computing environments

Enabling Smart Transportation Systems: A Parallel Spatio-Temporal Database Approach

Robust and Efficient Large-Large Table Outer Joins on Distributed Infrastructures

Contact Info

Product

Resources

About