Discovering large subsets with high quality partitions in real world graphs

Abstract

Given a real world graph, how can we find a large subgraph whose partition quality is much better than the original? Graph partitioning has received great attentions in graph mining, and especially balanced graph partitioning is required in many real world applications. However, the balanced graph partitioning is known to be NP-hard, and moreover it is known that there is no good cut at a large scale for real graphs. Due to this difficulty, in this paper, we propose a new paradigm for graph partitioning. Instead of dealing with the whole graph, our focus is on finding a large subgraph with high quality partitions, in terms of conductance. We show that removing problematic nodes, i.e. large degree nodes called hub nodes in real graphs, remarkably decreases conductance for the remaining giant connected component (GCC), while the number of nodes in the GCC is still significant. In experiments, we demonstrate that our method finds a subgraph of quite a large size with low conductance graph partitions, compared with competing methods. We also show that the competitors cannot find connected subgraphs while our method does, by construction. This improvement in partition quality for the subgraph is especially noticeable for large scale cuts-for a balanced partition, down to 14% of the original conductance with GCC size 70% of the total. As a result, the found subgraph has clear partitions at almost all scales compared with the original, and this result especially helps find communities which are well-formed, but hidden by hubs at various scales in real world graphs like social networks.

PACC: Large scale connected component computation on Hadoop and Spark

“…Each minimum reachable node represent a connected component. (5,5), (11,5), (3,3), (6,3), (12,3)}. For a node u, Γ(u) = {v|(u, v) 2 E} is the set of u's neighbors.…”

“…A connected component in a graph is a set of nodes linked to each other by paths. Finding connected components is a fundamental graph mining task having various applications including reachability [1,2], pattern recognition [3,4], graph partitioning [5,6], random walk [7], graph compression [8,9], etc. However, graphs of interest are enormous with billions of nodes and edges; e.g., 2.4 billion monthly active users form a huge friendship network in Facebook https://newsroom.fb.com/company-info/ and the Web is a gigantic network where more than 1 trillion Web pages are linked to each other by hyperlinks https://googleblog.blogspot.…”

A meta-graph approach to analyze subgraph-centric distributed programming models

“…When performing the second level of vertex-balanced partitioning within each machine, the subgraph containing the star-topology vertex may need to be split to ensure vertex balancing. This can result in partitions containing thousands of singleton or tiny subgraphs which are connected by remote edges to a subgraph containing the high edge-degree vertex present in another partition in the same machine [27]. So, for HP, the number of meta-vertices | V | will increase much more rapidly than FP, relative to DP and also as the number of machines or cores increases.…”

GraphZeppelin: Storage-Friendly Sketching for Connected Components on Dynamic Graph Streams

“…A recent survey by Sahu et al [66] of industrial uses of algorithms reports that, for both practitioners and academic researchers, connected components was the most frequently performed computation from a list of 13 fundamental graph problems that includes shortest paths, triangle counting, and minimum spanning trees. It has applications in scientific computing [62,69], flow simulation [70], metagenome assembly [27,58], identifying protein families [53,76], analyzing cell networks [5], pattern recognition [31,38], graph partitioning [46,47], random walks [36], social network community detection [42], graph compression [37,45], medical imaging [33], and object recognition [32]. It is a starting point for strictly harder problems such as edge/vertex connectivity, shortest paths, and 𝑘-cores.…”