Scale-Free Graph Processing on a NUMA Machine

Abstract

The importance of high-performance graph processing to solve big data problems targeting high-impact applications is greater than ever before. Graphs incur highly irregular memory accesses which leads to poor data locality, load imbalance, and data-dependent parallelism. Distributed graph processing frameworks, such as Google's Pregel, that employs memory-parallel, shared-nothing systems have experienced tremendous success in terms of scale and performance. Modern sharedmemory systems embrace the so called Non-Uniform Memory Access (NUMA) architecture which has proven to be more scalable (in terms of numbers of cores and memory modules) than the Symmetric Multiprocessing (SMP) architecture. In many ways, a NUMA system resembles a shared-nothing distributed system: physically distinct processing cores and memory regions (although, cache-coherent in NUMA). Memory accesses to remote NUMA domains are more expensive than local accesses. This poses the opportunity to transfer the know-how and design of distributed graph processing to develop shared-memory graph processing solutions optimized for NUMA systems (which is surprisingly little-explored).In this dissertation, we explore if a distributed-memory like middleware that makes graph partitioning and communication between partitions explicit, can improve the performance on a NUMA system. We design and implement a NUMA aware graph processing framework that treats the NUMA platform as a distributed system, and embraces its design principles; in particular explicit partitioning and inter-partition communication. We further explore design trade-offs to reduce communication overhead and propose a solution that embraces design philosophies of distributed graph processing system and at the same time exploits optimization opportunities specific to single-node systems. We demonstrate up to 13.9× speedup iii over a state-of-the-art NUMA-aware framework, Polymer and up to 3.7× scalability on a four-socket machine using graphs with tens of billions of edges.iv Preface This thesis is based on the research project done by me under the supervision and guidance of Professor Matei Ripeanu. I was responsible for the design, implementation, modeling, validation, evaluation and analysis of the results, along with taking the lead in publication writing effort. The research presented in this thesis have been either published or accepted for publication.The work that this thesis extends and evaluates against, was selected based on the following preliminary study; Professor Ripeanu and Tahsin helped me in the analysis of the results and editing the publication.