PathGraph: A Path Centric Graph Processing System

Yuan, Pingpeng; Xie, Changfeng; Liu, Ling; Jin, Hai

doi:10.1109/tpds.2016.2518664

Cited by 30 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior work has proposed several preprocessing algorithms to reorder sparse data structures to improve locality [20,54,59,64,65,67]. For example, graph preprocessing techniques reorder vertices in memory so that closely related vertices are stored nearby.…”

Section: Preprocessing Algorithmsmentioning

confidence: 99%

Phi

Mukkara¹,

Beckmann²,

Sánchez³

2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Many applications perform frequent scatter update operations to large data structures. For example, in push-style graph algorithms, processing each vertex requires updating the data of all its neighbors. Neighbors are often scattered over the whole graph, so these scatter updates have poor spatial and temporal locality. In current systems, scatter updates suffer high synchronization costs and high memory traffic. These drawbacks make push-style execution unattractive, and, when algorithms allow it, programmers gravitate towards pull-style implementations based on gather reads instead. We present PHI, a push cache hierarchy that makes scatter updates synchronization-and bandwidth-efficient. PHI adds support for pushing sparse, commutative updates from cores towards main memory. PHI adds simple compute logic at each cache level to buffer and coalesce these commutative updates throughout the hierarchy. This avoids synchronization, exploits temporal locality, and produces a load-balanced execution. Moreover, PHI exploits spatial locality by selectively deferring updates with poor spatial locality, batching them to achieve sequential main memory transfers. PHI is the first system to leverage both the temporal and spatial locality benefits of commutative scatter updates, some of which do not apply to gather reads. As a result, PHI not only makes push algorithms efficient, but makes them consistently faster than pull ones. We evaluate PHI on graph algorithms and other sparse applications processing large inputs. PHI improves performance by 4.7× on average (and by up to 11×), and reduces memory traffic by 2× (and by up to 5×). CCS CONCEPTS • Computer systems organization → Multicore architectures.

show abstract

Section: Preprocessing Algorithmsmentioning

confidence: 99%

Phi

Mukkara¹,

Beckmann²,

Sánchez³

2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

show abstract

“…The most famous representative is MapReduce [18]. According to the kinds of programming models adopted, dedicated big graph processing systems can be classified into TLAV (Think Like A Vertex) [14], [15], [19] ones, TLAE (Think Like An Edge) [20] ones, TLAP (Think Like A Path) [21] ones, TLAC (Think Like A Component) [22] ones, TLAB (Think Like A Block) [23] ones, and TLAG (Think Like A subGraph) [17] ones. For all these systems [14], [15], [17], [20]- [23], users only need to program procedures for processing a single vertex, edge, path, component, block, or subgraph through programming interfaces provided by these systems, respectively.…”

Section: A Current Graph Processing Systemsmentioning

confidence: 99%

“…Big graph parallel processing is the use of multiple computational nodes of a parallel computing system to process a big graph for the purpose of speeding up of processing. The common processing flow shared among most graph processing systems are: (1) they load a big graph from a huge-capacity storage device and partition it into a series of subgraphs with approximately equal scale, which are then distributed to different nodes of the system, and (2) they employ TLAV [14], [15], [19], TLAE [20], TLAP [21], TLAC [22], TLAB [23], or TLAG [17], through which users program procedures expressing their graph algorithms. These procedures are the scheduling units for parallel processing.…”

Section: B Traditional Parallel Processing Flow Of Big Graphsmentioning

confidence: 99%

Cluster-Scheduling Big Graph Traversal Task for Parallel Processing in Heterogeneous Cloud Based on DAG Transformation

Zeng

Ding

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Task scheduling is the key to the full utilization of heterogeneous cloud capabilities for parallel processing of big graphs. Most graph processing systems adopt single-granularity scheduling mechanisms without considering the heterogeneity of the cloud, leading to poor performance. To alleviate it by learning from the excellent directed acyclic graph (DAG)-based scheduling techniques accumulated in traditional parallel computing, we first present a streaming DAG-construction heuristic. It transforms a big graph along with graph traversal algorithms to be carried out into a DAG. We then propose a three-phase heterogeneousaware cluster-scheduling algorithm to schedule the DAG into a heterogeneous cloud for parallel processing. In the first phase, we design a parallel linear clustering algorithm to cluster the DAG into a series of linear clusters with different granularities. In the second phase, we design a heterogeneous-aware load balancing algorithm to map these clusters to different computational nodes of the cloud. In the last phase, we design a task ordering algorithm to assigns these clusters as-early-as-possible start times. The experimental results show that our scheme can generate high-quality schedules and improve the efficiency and performance of parallel processing of big graphs in the heterogeneous cloud. INDEX TERMS Heterogeneous cloud, big graph traversal task, parallel processing, DAG, clusterscheduling, granularity.

show abstract

“…In addition, labeled and unlabeled graphs can be processed, too. Since undirected graph can be easily transformed into directed graph by adding another edge between two connected vertices, the following discussion mainly focuses on directed connected graph defined in [5,6].…”

Section: Preliminarymentioning

confidence: 99%

“…The existing algorithms may be grouped in two divisions: edge cut algorithms and vertex-cut algorithms. The majority of distributed graph engines adopt edge-based hash partitioning [3][4][5][6] as the data partitioning solution. Edge-based hash partitioning is a vertex-cut approach which distributes edges across the partitions by computing the hash keys of vertices and allows edges to span partitions.…”

Section: Introductionmentioning

confidence: 99%

An Association-Oriented Partitioning Approach for Streaming Graph Query

Hao

Yuan

et al. 2017

Scientific Programming

Self Cite

View full text Add to dashboard Cite

The volumes of real-world graphs like knowledge graph are increasing rapidly, which makes streaming graph processing a hot research area. Processing graphs in streaming setting poses significant challenges from different perspectives, among which graph partitioning method plays a key role. Regarding graph query, a well-designed partitioning method is essential for achieving better performance. Existing offline graph partitioning methods often require full knowledge of the graph, which is not possible during streaming graph processing. In order to handle this problem, we propose an association-oriented streaming graph partitioning method named Assc. This approach first computes the rank values of vertices with a hybrid approximate PageRank algorithm. After splitting these vertices with an adapted variant affinity propagation algorithm, the process order on vertices in the sliding window can be determined. Finally, according to the level of these vertices and their association, the partition where the vertices should be distributed is decided. We compare its performance with a set of streaming graph partition methods and METIS, a widely adopted offline approach. The results show that our solution can partition graphs with hundreds of millions of vertices in streaming setting on a large collection of graph datasets and our approach outperforms other graph partitioning methods.

show abstract

PathGraph: A Path Centric Graph Processing System

Cited by 30 publications

References 20 publications

Phi

Phi

Cluster-Scheduling Big Graph Traversal Task for Parallel Processing in Heterogeneous Cloud Based on DAG Transformation

An Association-Oriented Partitioning Approach for Streaming Graph Query

Contact Info

Product

Resources

About