Linear work generation of R-MAT graphs

Abstract

R-MAT (for Recursive MATrix) is a simple, widely used model for generating graphs with a power law degree distribution, a small diameter, and communitys structure. It is particularly attractive for generating very large graphs because edges can be generated independently by an arbitrary number of processors. However, current R-MAT generators need time logarithmic in the number of nodes for generating an edge— constant time for generating one bit at a time for node IDs of the connected nodes. We achieve constant time per edge by precomputing pieces of node IDs of logarithmic length. Using an alias table data structure, these pieces can then be sampled in constant time. This simple technique leads to practical improvements by an order of magnitude. This further pushes the limits of attainable graph size and makes generation overhead negligible in most situations.

Engineering Massively Parallel MST Algorithms

“…The number of (symmetric, directed) edges of these graphs ranges from 57 millions to 123 billions. In our weak scaling experiments, we use instances of six different graph families generated with KaGen 5 [52] and a fast RMAT generator [53]. We use two-dimensional grids (2D-GRID), two-and threedimensional random geometric graphs (2D/3D-RGG), random hyperbolic graphs (RHG), Erdős-Renyi graphs (GNM) and RMAT graphs.…”

“…KaGen ensures that the generated edges are globally lexicographically sorted and thus do not produce shared vertices for the input. Regarding the RMAT generator [53], we first globally sort the generated edges and then redistribute them equally over all PEs. For MND-MST, the edges incident to a shared vertex are moved completely to one MPI process to meet their input format.…”

R3MAT: A Rapid and Robust Graph Generator

“…In [33], the authors present a method to reduce the R-MAT algorithm from logarithmic to constant time per edge. The basic idea is that a logarithmic number of address bits can be generated in each iteration without changing the underlying process.…”

“…In contrast, we developed a sequential implementation that allows to generate large graphs in small-memory machines, with a reasonable runtime. Finally, [33] gives an algorithm to quickly generate a graph with R-MAT, but it suffers from the same drawbacks of the other alternatives.…”

Scalable generation of random graphs

“…The time for generating an edge can be reduced from logarithmic to constant using bit parallelism [181]. The algorithm precomputes sequences of a logarithmic number of decisions together with their probability.…”