Fast Iterative Graph Computation: A Path Centric Approach

Yuan, Pingpeng; Zhang, Wenya; Xie, Changfeng; Jin, Hai; Liu, Ling; Lee, Kisung

doi:10.1109/sc.2014.38

Cited by 66 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, labeled and unlabeled graphs can be processed, too. Since undirected graph can be easily transformed into directed graph by adding another edge between two connected vertices, the following discussion mainly focuses on directed connected graph defined in [5,6].…”

Section: Preliminarymentioning

confidence: 99%

“…The existing algorithms may be grouped in two divisions: edge cut algorithms and vertex-cut algorithms. The majority of distributed graph engines adopt edge-based hash partitioning [3][4][5][6] as the data partitioning solution. Edge-based hash partitioning is a vertex-cut approach which distributes edges across the partitions by computing the hash keys of vertices and allows edges to span partitions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Association-Oriented Partitioning Approach for Streaming Graph Query

Hao

Yuan

et al. 2017

Scientific Programming

Self Cite

View full text Add to dashboard Cite

The volumes of real-world graphs like knowledge graph are increasing rapidly, which makes streaming graph processing a hot research area. Processing graphs in streaming setting poses significant challenges from different perspectives, among which graph partitioning method plays a key role. Regarding graph query, a well-designed partitioning method is essential for achieving better performance. Existing offline graph partitioning methods often require full knowledge of the graph, which is not possible during streaming graph processing. In order to handle this problem, we propose an association-oriented streaming graph partitioning method named Assc. This approach first computes the rank values of vertices with a hybrid approximate PageRank algorithm. After splitting these vertices with an adapted variant affinity propagation algorithm, the process order on vertices in the sliding window can be determined. Finally, according to the level of these vertices and their association, the partition where the vertices should be distributed is decided. We compare its performance with a set of streaming graph partition methods and METIS, a widely adopted offline approach. The results show that our solution can partition graphs with hundreds of millions of vertices in streaming setting on a large collection of graph datasets and our approach outperforms other graph partitioning methods.

show abstract

Section: Preliminarymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Association-Oriented Partitioning Approach for Streaming Graph Query

Hao

Yuan

et al. 2017

Scientific Programming

Self Cite

View full text Add to dashboard Cite

show abstract

“…Given that GraphX requires Spark and larger memory to run, we extract the performance results of GraphX, GraphLab, and Giraph from [8], annotated with their respective system configurations for the same graph datasets. The results of Giraph++ are extracted from [28]. We offer a number of observations.…”

Section: Comparison With Existing Systemsmentioning

confidence: 99%

“…Disk-based systems focus on improving the performance of iterative computations on a single machine [13,20,9,28,30]. GraphChi [13] is based on the vertex-centric model.…”

Section: Related Workmentioning

confidence: 99%

Scaling iterative graph computations with GraphMap

Lee

Liu

Schwan

et al. 2015

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

In recent years, systems researchers have devoted considerable effort to the study of large-scale graph processing. Existing distributed graph processing systems such as Pregel, based solely on distributed memory for their computations, fail to provide seamless scalability when the graph data and their intermediate computational results no longer fit into the memory; and most distributed approaches for iterative graph computations do not consider utilizing secondary storage a viable solution. This paper presents GraphMap, a distributed iterative graph computation framework that maximizes access locality and speeds up distributed iterative graph computations by effectively utilizing secondary storage. GraphMap has three salient features: (1) It distinguishes data states that are mutable during iterative computations from those that are read-only in all iterations to maximize sequential access and minimize random access. (2) It entails a two-level graph partitioning algorithm that enables balanced workloads and locality-optimized data placement. (3) It contains a proposed suite of locality-based optimizations that improve computational efficiency. Extensive experiments on several real-world graphs show that GraphMap outperforms existing distributed memory-based systems for various iterative graph algorithms.

show abstract

“…However, since developing distributed graph algorithm is challenging, some researchers divert their attention to design the graph processing system that handle large scale graphs on a single PC. The research endeavours in this direction have delivered the systems such as GraphChi [17], PathGraph [45], GraphQ [39], LLAMA [27] and GridGraph [51]. However, these systems suffer from the limited degree of parallelism in conventional processors.…”

Section: Introductionmentioning

confidence: 99%

WolfPath: Accelerating Iterative Traversing-Based Graph Processing Algorithms on GPU

Zhu

et al. 2017

Int J Parallel Prog

View full text Add to dashboard Cite

There is the significant interest nowadays in developing the frameworks of parallelizing the processing for the large graphs such as social networks, Web graphs, etc. Most parallel graph processing frameworks employ iterative processing model. However, by benchmarking the state-of-art GPU-based graph processing frameworks, we observed that the performance of iterative traversing-based graph algorithms (such as Bread First Search, Single Source Shortest Path and so on) on GPU is limited by the frequent data exchange between host and GPU. In order to tackle the problem, we develop a GPU-based graph framework called WolfPath to accelerate the processing of iterative traversing-based graph processing algorithms. In WolfPath, the iterative process is guided by the graph diameter to eliminate the frequent data exchange between host and GPU. To accomplish this goal, WolfPath proposes a data structure called Layered Edge list to represent the graph, from which the graph diameter is known before the start of graph processing. In order to enhance the applicability of our WolfPath 123Int J Parallel Prog framework, a graph preprocessing algorithm is also developed in this work to convert any graph into the format of the Layered Edge list. We conducted extensive experiments to verify the effectiveness of WolfPath. The experimental results show that WolfPath achieves significant speedup over the state-of-art GPU-based in-memory and out-of-memory graph processing frameworks.

show abstract

Fast Iterative Graph Computation: A Path Centric Approach

Cited by 66 publications

References 27 publications

An Association-Oriented Partitioning Approach for Streaming Graph Query

An Association-Oriented Partitioning Approach for Streaming Graph Query

Scaling iterative graph computations with GraphMap

WolfPath: Accelerating Iterative Traversing-Based Graph Processing Algorithms on GPU

Contact Info

Product

Resources

About