PageRank Pipeline Benchmark: Proposal for a Holistic System Benchmark for Big-Data Platforms

Abstract

The rise of big data systems has created a need for benchmarks to measure and compare the capabilities of these systems. Big data benchmarks present unique scalability challenges. The supercomputing community has wrestled with these challenges for decades and developed methodologies for creating rigorous scalable benchmarks (e.g., HPC Challenge). The proposed PageRank pipeline benchmark employs supercomputing benchmarking methodologies to create a scalable benchmark that is reflective of many real-world big data processing systems. The PageRank pipeline benchmark builds on existing prior scalable benchmarks (Graph500, Sort, and PageRank) to create a holistic benchmark with multiple integrated kernels that can be run together or independently. Each kernel is well defined mathematically and can be implemented in any programming environment. The linear algebraic nature of PageRank makes it well suited to being implemented using the GraphBLAS standard.The computations are simple enough that performance predictions can be made based on simple computing hardware models. The surrounding kernels provide the context for each kernel that allows rigorous definition of both the input and the output for each kernel. Furthermore, since the proposed PageRank pipeline benchmark is scalable in both problem size and hardware, it can be used to measure and quantitatively compare a wide range of present day and future systems. Serial implementations in C++, Python, Python with Pandas, Matlab, Octave, and Julia have been implemented and their single threaded performance has been measured.

Static graph challenge: Subgraph isomorphism

“…1) Benchmarking on Synthetic Graph Data: The multilanguage implementations of the triangle counting and ktruss algorithms were benchmarked on synthetic graphs. The graphs were generated as M xM images where M = 2 n , n = 8, 9,10,11,12,13. Each pixel in the image was treated as a node in the graph.…”

In-storage embedded accelerator for sparse pattern processing

“…We would like to explore the application of in-storage analytics accelerator in a cloud [20] [21] or supercomputing setting. We are also currently developing GraphBLAS compliant operations in our system for common graph and sparse linear algebra problems such as one proposed in [22]. We would also like to investigate how to integrate our system into more general data management solutions such as the BigDAWG polystore system [23].…”

GraphChallenge.org: Raising the Bar on Graph Analytic Performance

“…The Graph Challenge consists of three challenges • Pre-challenge: PageRank pipeline [58] • Static graph challenge: subgraph isomorphism [59] • Streaming graph challenge: stochastic block partition [60] The static graph challenge is further broken down into triangle counting and k-truss. Triangle counting received the most submissions and is the focus of this paper.…”