LiteTE: Lightweight, Communication-Efficient Distributed-Memory Triangle Enumerating

Abstract

Distributed-memory triangle enumerating has attracted considerable interests due to its potential capability to process huge graphs quickly. However, existing algorithms suffer from low speed due to high communication cost and load imbalance. To solve the problems, we propose LiteTE, a lightweight, communication-efficient triangle enumerating scheme. To reduce communication cost, LiteTE proposes several techniques, including a graph partitioning method to fully leverage the large memory of commodity servers and the high bandwidth of modern networks and a fast broadcast algorithm to effectively utilize the bidirectional bandwidth of cables and the aggregate bandwidth of clusters. To reduce load imbalance, LiteTE proposes three-level techniques, including a codesign technique of graph partitioning and partition-level load balance, a decentralized dynamic node-level load balance technique, and a chunk-based lock-free work-stealing technique, all of which are lightweight and incur no or hardly any communication cost. The experimental results show that LiteTE reduces communication cost and load imbalance considerably and achieves much better performance in metrics, such as setup time, runtime, scalability, and load balance than the state-of-the-art algorithms. On a small-scale cluster, LiteTE enumerates the 15 trillion triangles in a graph of 92 billion edges in 15 min, while other algorithms fail to complete. INDEX TERMS Triangle enumerating, triangle computation, graph processing, distributed computing, parallel processing.

TRUST: Triangle Counting Reloaded on GPUs

“…This method, however, would require interworker communication when counting triangles. Recently, LiteTe [42] also attempts to use value-range based 2D partitioning for triangle counting, but, again, experiences workload imbalance.…”

“…In order to achieve better performance, both designs need communication free and workload balanced graph partitioning. However, even for edge-centric design which only requires 1-hop neighbor lists, achieving both goals is challenging, which is evident both theoretically [22], [41] and practically [19], [22], [23], [27], [42]. The vertex-centric design requires 2-hop neighbors, which further exacerbates the imbalance and communication problems.…”

“…Particularly, both time IR and space IR lie between 1 and 1.1 for both graphs on 1,024 GPUs. In contrast, LiteTe [42], which uses range-based partitioning scheme, has a much higher time IR, i.e., 1.7 for UK graph.…”

TriPoll: Computing Surveys of Triangles in Massive-Scale Temporal Graphs with Metadata

“…This alleviates the imbalances in triangle identification to a point that cyclic partitioning becomes palatable. Previous work has shown partitioning to be helpful, specifically if data replication is allowed [39,64].…”

Parallel Algorithms for Butterfly Computations

“…There have also been various methods for approximating subgraph counts via sampling [34,3,4,63,32,13,50,42]. Finally, there has also been significant work for the past decade on parallel triangle counting algorithms (e.g., [56,7,57,6,46,47,15,45,58,60,39,35,26,25,30,19,28,65] and papers from the annual GraphChallenge [24], among many others).…”