Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis

Abstract

Graph-theoretic abstractions are extensively used to analyze massive data sets. Temporal data streams from socioeconomic interactions, social networking web sites, communication traffic, and scientific computing can be intuitively modeled as graphs. We present the first study of novel highperformance combinatorial techniques for analyzing largescale information networks, encapsulating dynamic interaction data in the order of billions of entities. We present new data structures to represent dynamic interaction networks, and discuss algorithms for processing parallel insertions and deletions of edges in small-world networks. With these new approaches, we achieve an average performance rate of 25 million structural updates per second and a parallel speedup of nearly 28 on a 64-way Sun UltraSPARC T2 multicore processor, for insertions and deletions to a small-world network of 33.5 million vertices and 268 million edges. We also design parallel implementations of fundamental dynamic graph kernels related to connectivity and centrality queries. Our implementations are freely distributed as part of the open-source SNAP (Small-world Network Analysis and Partitioning) complex network analysis framework.

Fast Computing Betweenness Centrality with Virtual Nodes on Large Sparse Networks

“…As a result, many large-scale networks are regarded as unweighted when the above measures are reported [2] , [3] . Large efforts have been made to improve the efficiency of algorithms for calculating those network properties [10] , [11] . Take the betweenness centrality, for example [12] , [13] : for a weighted network with nodes and edges, the naive algorithm requires time and storage, regardless of the algorithms implemented to find the shortest paths.…”

“…A much faster algorithm proposed by Brandes [14] , on the other hand, can calculate the betweenness centrality in time and space when the shortest paths are calculated by Dijkstra's algorithm implemented with a Fibonacci heap. Parallel algorithms are also proposed to improve the efficiency for the calculation of betweenness centrality [10] , [11] , [15] – [21] : for example, Bader and Madduri [10] proposed a betweenness centrality algorithm on a high-end shared memory symmetric multiprocessor and multithreaded architectures, with which is “possible” to achieve the computation in time with access conflicts, where is the number of processors used. However, the parallel algorithms requires much more complex programming and are highly dependent on the hardwares: for example, in Bader and Madduri's study [10] , they used an IBM p5 570 on 16 processors and utilized 20GB RAM.…”

Massive streaming data analytics: A case study with clustering coefficients

“…Alternative graph data structures include forms of binary trees [18]. Trees pay an extra cost in keeping some order on the edges.…”

Parallelism in simulation and modeling of scale-free complex networks

“…Parallel algorithms for generation of small-world networks has been studied in [2,15]. Small-world networks have similar features as scale-free networks (for example small diameter) but the generating stochastic processes do not use preferential selection which is the main obstacle to the parallelization of scale-free networks as we argue below.…”