HyGN: Hybrid Graph Engine for NUMA

Aasawat, Tanuj Kr; Reza, Tahsin; Yoshizoe, Kazuki; Ripeanu, Matei

doi:10.1109/bigdata50022.2020.9378430

Cited by 7 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Voronoi cell computation naturally inclines to vertex-centric parallel processing [23]. Computation of a single cell closely resembles single source shortest path (SSSP) computation and Bellman-Ford based fast, vertex parallel SSSP algorithms are well known [23], [24]. It is worth noting that Ceccarello et al [25] used the work-efficient ∆-Stepping algorithm in parallel shortest path computation from multiple sources for diameter approximation of weighted graphs (comparable to Voronoi cell computation).…”

Section: Parallel Steiner Tree Algorithmmentioning

confidence: 99%

“…As highlighted in the time complexity analysis ( §III), Voronoi cell computation is the most expensive step in our algorithm, therefore, we seek a solution that accelerates the throughput of this step. Previous studies [24], [27] showed that asynchronous processing offers notable advantage over bulk synchronous processing (BSP) for distributed shortest path computation: the former enabling faster convergence. To this end, we begin with HavoqGT [19] as the foundation and implement other features required by Alg.…”

Section: Distributed Implementationmentioning

confidence: 99%

“…Edge weight distribution is known to influence the convergence time of shortest path computation [24], [27]. Fig.…”

Section: Edge Weight Distribution Vs Runtime Performancementioning

confidence: 99%

See 2 more Smart Citations

Towards Distributed 2-Approximation Steiner Minimal Trees in Billion-edge Graphs

Reza¹,

Sanders²,

Pearce³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Given an edge-weighted graph and a set of known seed vertices of interest, a network scientist often desires to understand the graph relationships to explain connections between the seed vertices. If the size of the seed set is 2, shortest path calculations are an attractive computational kernel to explore the connections between the two vertices. When the seed set is 3 or larger (say up to 1,000s) Steiner minimal tree -min-weight acyclic connected subgraph (of the input graph) that contains all the seed vertices -is an attractive generalization of shortest weighted paths. In general, computing a Steiner minimal tree is NP-hard, but decades ago several polynomial-time algorithms were designed and proven to yield Steiner trees whose total weight is bounded within 2 times the minimal Steiner tree.Despite its rich theoretical literature, works related to parallel Steiner minimal tree computation and their scalable implementations are rather scarce. In this paper, we present a parallel 2approximation Steiner minimal tree algorithm (with theoretical guarantees) and its MPI-based distributed implementation. In place of distance computation between all pairs of seed vertices, an expensive phase in many approximation algorithms, the solution we employ, exploits Voronoi cell computation. Also, this approach has higher parallel efficiency than others that involve minimum spanning tree computation on the entire graph. Furthermore, our distributed design exploits asynchronous processing and a message prioritization scheme to accelerate convergence of distance computation, employs techniques to avoid inefficient distributed spanning tree computation on the entire graph, and harnesses a combination of vertex and edge centric processing to offer fast time-to-solution. We demonstrate scalability and performance of our solution using real-world graphs with up to 128 billion edges and 512 compute nodes (8K processes), show the ability to find Steiner trees with up to 10K seed vertices in under one minute, and present in-depth analyses that highlight the benefits of our design choices. Using four real-world graphs and three seed sets for each, we compare our solution with the state-of-the-art exact Steiner minimal tree solver, SCIP-Jack, and two sequential algorithms with the same approximation bound as our algorithm. Our distributed solution comfortably outperforms these related works on graphs with 10s million edges and offers decent strong scaling -up to 90% efficient. We empirically show that, on average, the total distance (sum of edge weights) of the Steiner tree identified by our solution is 1.0527 times greater than the Steiner minimal tree (i.e., the optimal solution) -well within the theoretical bound of less than equal to 2.

show abstract

Section: Parallel Steiner Tree Algorithmmentioning

confidence: 99%

Section: Distributed Implementationmentioning

confidence: 99%

See 1 more Smart Citation

Towards Distributed 2-Approximation Steiner Minimal Trees in Billion-edge Graphs

Reza¹,

Sanders²,

Pearce³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This framework uses thread‐based partitioning to distribute computation and communication with threads on a machine cluster while it removes the requirement of needless thread synchronizations. Inspired by the shared‐nothing scale‐out designs of distributed graph processing frameworks, Aasawat et al 54 proposed the Hybrid Graph processing engine for NUMA (HYGN): a graph processing engine that exploits the characteristics of the synchronous and asynchronous processing modes. Their solution partitions the graph and binds each partition to a NUMA node to maximize locality.…”

Section: Related Workmentioning

confidence: 99%

Improving the efficiency of graph algorithm executions on high‐performance computing

Moori

Rocha

Schwarzrock

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

The growing need for extracting information from large graphs has been pushing the development of parallel graph algorithms. However, the highly irregular structure of the real-world graphs limits the performance and energy improvements of graph applications. In this paper, we show that, in most cases, using all the available cores of the multiprocessor is not the best option in terms of the aforementioned non-functional requirements. Based on that, we propose GraphKat, a framework that enables the simultaneous processing of several algorithms/graphs instead of executing them serially (i.e., one after another), increasing efficiency in terms of performance and energy.GraphKat works in two steps: (i) it characterizes the graph applications with a specific number of threads based on their efficiency levels; and (ii) it defines the execution order of all graph applications in the target system. Experimental results on three multicore processors (Intel and AMD) show that GraphKat improves the overall system's efficiency related to performance (up to 434.26×) and energy-saving (up to 245.21×), and reduces the graph applications' execution time (up to 17.70×) and energy consumption (up to 6.64×) compared to the default execution of parallel applications on HPC systems.

show abstract

Performance Optimization for Parallel SpMV on a NUMA Architecture

Zhang

et al. 2022

Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery

View full text Add to dashboard Cite

HyGN: Hybrid Graph Engine for NUMA

Cited by 7 publications

References 14 publications

Towards Distributed 2-Approximation Steiner Minimal Trees in Billion-edge Graphs

Towards Distributed 2-Approximation Steiner Minimal Trees in Billion-edge Graphs

Improving the efficiency of graph algorithm executions on high‐performance computing

Performance Optimization for Parallel SpMV on a NUMA Architecture

Contact Info

Product

Resources

About