A Distributed Multilevel Force-Directed Algorithm

Abstract

The wide availability of powerful and inexpensive cloud computing services naturally motivates the study of distributed graph layout algorithms, able to scale to very large graphs. Nowadays, to process Big Data, companies are increasingly relying on PaaS infrastructures rather than buying and maintaining complex and expensive hardware. So far, only a few examples of basic force-directed algorithms that work in a distributed environment have been described. Instead, the design of a distributed multilevel force-directed algorithm is a much more challenging task, not yet addressed. We present the first multilevel force-directed algorithm based on a distributed vertex-centric paradigm, and its implementation on Giraph, a popular platform for distributed graph algorithms. Experiments show the effectiveness and the scalability of the approach. Using an inexpensive cloud computing service of Amazon, we draw graphs with ten million edges in about 60 minutes.

Cornac: Tackling Huge Graph Visualization with Big Data Infrastructure

“…For graphs more specifically, the Big Data ecosystem has been used for graph drawing. Arleo et al [19] implemented a spring-embedder [20] on top of Giraph and then a multilevel force-directed algorithm [21]. Their algorithm uses edges to pass information about neighbors, using Pregel.…”

“…It exists a rich litterature on large graph drawing. For instance, Hadany and Harel [24] introduced the basis of multi-scale algorithm, [25], [26], [27] presented several distributed algorithms and [19], [21], [22] presented the first working algorithms on Big Data infrastructure. We refer the reader to the handbook of graph drawing [28] for more information on that topic.…”

GiViP: A Visual Profiler for Distributed Graph Processing Systems

“…This feature can be used both for a visual confirmation of a successful execution and for didactic purposes. We considered four algorithms: Single-Source Shortest-Path (SSSP) and Page Rank (PR) [42] are well-known graph algorithms, available in the set of examples provided by the Apache Giraph library; GILA (GI) [7] and MultiGILA (MGI) [8] are TLAV implementations of a force-directed algorithm and of a multilevel forcedirected algorithm, respectively. We ran these algorithms on two graphs, cti and Gnutella31.…”

“…Also, in the final sifting procedure each vertex can be moved only within the subsequence of its host. While developing the GI and MGI algorithms [7,8], we were able to dramatically reduce the running times of these algorithms by replacing the default hash-based partitioning strategy of Giraph with a more sophisticated one, called Spinner [62]. The algorithm in [62] computes partitions with increased locality, i.e., such that the number of edges in the input graph having their end-vertices in different workers is minimized.…”

“…In view of their effectiveness, these systems are being adopted by a growing number of applications. For example Apache Giraph is used in the contexts of social networking [36], fraud detection [59], and network visualization [7,8]. While many graph processing systems working on top of modern distributed infrastructures have been proposed to deal with large graphs, the tasks of profiling and debugging their massive computations remain time consuming and error-prone [30,53].…”