An experimental comparison of pregel-like graph processing systems

Abstract

The introduction of Google's Pregel generated much interest in the field of large-scale graph data processing, inspiring the development of Pregel-like systems such as Apache Giraph, GPS, Mizan, and GraphLab, all of which have appeared in the past two years. To gain an understanding of how Pregel-like systems perform, we conduct a study to experimentally compare Giraph, GPS, Mizan, and GraphLab on equal ground by considering graph and algorithm agnostic optimizations and by using several metrics. The systems are compared with four different algorithms (PageRank, single source shortest path, weakly connected components, and distributed minimum spanning tree) on up to 128 Amazon EC2 machines. We find that the system optimizations present in Giraph and GraphLab allow them to perform well. Our evaluation also shows Giraph 1.0.0's considerable improvement since Giraph 0.1 and identifies areas of improvement for all systems.

Let High-level Graph Queries Be Parallel Efficient: An Approach Over Structural Recursion On Pregel

“…However, when we increase the number of processors up to 64, we could not gain linear speedup, but the execution time still decreases. This phenomenon has also been observed in other Pregel-based frameworks [6], [10]. The exact reasons of this limitation shall be studied in the future, but it is out of the scope of this paper.…”

From Think Parallel to Think Sequential

“…The communication cost (in bytes) reported by Giraph, GraphLab and Blogel depends on their own implementation of message blocks and protocols [19]. For a fair comparison, we tracked the total bytes sent by each machine during a run, by monitoring the system file /proc/net/dev, following [19].…”

“…Trinity and c ACM 2017. This is a minor revision of the paper entitled Parallelizing Sequential Graph Computations, published in SIGMOD'17, ISBN978-1-4503-4197-4/17/05, May [14][15][16][17][18][19]2017, Chicago, Illinois, USA. DOI: http://dx.doi.org/10.1145/3035918.3035942.…”

Big Graph Analytics Platforms

“…Researchers new to this young field can easily get overwhelmed and lost by the large amount of literature. Although several experimental studies have been conducted in recent years that compare the performance of several big graph systems , Han et al, 2014a, Satish et al, 2014, Guo et al, 2014, there lacks a comprehensive survey that clearly summarizes the key features and techniques developed in existing big graph systems. A recent survey [McCune et al, 2015] attempts to cover the landscape as well, but primarily focuses on vertex-centric systems; it omits most of the work on other programming models and also several crucial optimization and programmability issues with vertex-centric systems.…”