A scalable distributed graph partitioner

Margo, Daniel; Seltzer, Margo

doi:10.14778/2824032.2824046

Cited by 62 publications

(41 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use Metis [12] as our primary basis for comparison because, despite its age, it remains a gold standard for producing quality workload-agnostic partitionings of medium sized graphs [16,28]. As such it is a compelling yard-stick for our evaluation of the TAPER prototype, which will consider partitioning quality in terms of scale free metrics such as i pt and the number of vertex swaps.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

TAPER: query-aware, partition-enhancement for large, heterogenous graphs

Firth

Missier

2017

Distrib Parallel Databases

View full text Add to dashboard Cite

Graph partitioning has long been seen as a viable approach to addressing Graph DBMS scalability. A partitioning, however, may introduce extra query processing latency unless it is sensitive to a specific query workload, and optimised to minimise inter-partition traversals for that workload. Additionally, it should also be possible to incrementally adjust the partitioning in reaction to changes in the graph topology, the query workload, or both. Because of their complexity, current partitioning algorithms fall short of one or both of these requirements, as they are designed for offline use and as one-off operations. The TAPER system aims to address both requirements, whilst leveraging existing partitioning algorithms. TAPER takes any given initial partitioning as a starting point, and iteratively adjusts it by swapping chosen vertices across partitions, heuristically reducing the probability of inter-partition traversals for a given path queries workload. Iterations are inexpensive thanks to time and space optimisations in the underlying support data structures. We evaluate TAPER on two different large test graphs and over realistic query workloads. Our results indicate that, given a hash-based partitioning, TAPER reduces the number of inter-partition traversals by ∼80%; given an unweighted Metis partitioning, by ∼30%. These reductions are achieved within eight iterations and with the additional advantage of being workload-aware and usable online.

show abstract

Section: Discussionmentioning

confidence: 99%

“…For graph data, balanced graph partitioning has been exhaustively studied in literature since the 1970s [8,12,13,16,23,25,26,28,30], and a number of practical implementations are available [12,23]. We do not seek a new graph partitioning algorithm; rather, to propose a workload-driven method for improving partitions that already exist.…”

Section: Related Workmentioning

confidence: 99%

TAPER: query-aware, partition-enhancement for large, heterogenous graphs

Firth

Missier

2017

Distrib Parallel Databases

View full text Add to dashboard Cite

show abstract

“…Moreover, they take much more time to compute than many graph algorithms. Sheep [29] is a distributed graph partitioner that produces high quality edge partitions an order of magnitude faster than METIS. Alternatively, linear-time heuristics have been proposed.…”

Section: Related Workmentioning

confidence: 99%

Accelerating Graph Analytics by Utilising the Memory Locality of Graph Partitioning

Sun

Vandierendonck

Nikolopoulos

2017

2017 46th International Conference on Parallel Processing (ICPP)

View full text Add to dashboard Cite

This paper investigates how to improve the memory locality of graph-structured analytics on large-scale shared memory systems. We demonstrate that a graph partitioning where all in-edges for a vertex are placed in the same partition improves memory locality. However, realising performance improvement through such graph partitioning poses several challenges and requires rethinking the classification of graph algorithms and preferred data structures. We introduce the notion of medium-dense frontiers, a type of frontier that is sufficiently dense for a bitmap representation, yet benefits from an indexed graph layout. Using three types of frontiers, and three graph layout schemes optimized to each frontier type, we design an edge traversal algorithm that autonomously decides which type to use. The distinction of forward vs. backward graph traversal folds into this decision and need no longer be specified by the programmer. We have implemented our techniques in a NUMA-aware graph analytics framework derived from Ligra and demonstrate a speedup of up to 4.34× over Ligra and up to 2.93× over Polymer.

show abstract

“…Moreover, they take much more time to compute than many graph algorithms. Sheep [17] is a distributed graph partitioner that produces high quality edge partitions an order of magnitude faster than METIS. Alternatively, linear-time heuristics have been proposed.…”

Section: Further Related Workmentioning

confidence: 99%

GraphGrind

Sun

Vandierendonck

Nikolopoulos

2017

Proceedings of the International Conference on Supercomputing

View full text Add to dashboard Cite

We investigate how graph partitioning adversely affects the performance of graph analytics. We demonstrate that graph partitioning induces extra work during graph traversal and that graph partitions have markedly different connectivity than the original graph. By consequence, increasing the number of partitions reaches a tipping point after which overheads quickly dominate performance gains. Moreover, we show that the heuristic to balance CPU load between graph partitions by balancing the number of edges is inappropriate for a range of graph analyses. However, even when it is appropriate, it is sub-optimal due to the skewed degree distribution of social networks. Based on these observations, we propose GraphGrind, a new graph analytics system that addresses the limitations incurred by graph partitioning. We moreover propose a NUMA-aware extension to the Cilk programming language and obtain a scale-free yet NUMA-aware parallel programming environment which underpins NUMA-aware scheduling in GraphGrind. We demonstrate that GraphGrind outperforms state-of-the-art graph analytics systems for shared memory including Ligra, Polymer and Galois.

show abstract

A scalable distributed graph partitioner

Cited by 62 publications

References 18 publications

TAPER: query-aware, partition-enhancement for large, heterogenous graphs

TAPER: query-aware, partition-enhancement for large, heterogenous graphs

Accelerating Graph Analytics by Utilising the Memory Locality of Graph Partitioning

GraphGrind

Contact Info

Product

Resources

About