Co-clustering directed graphs to discover asymmetries and directional communities

Abstract

In directed graphs, relationships are asymmetric and these asymmetries contain essential structural information about the graph. Directed relationships lead to a new type of clustering that is not feasible in undirected graphs. We propose a spectral co-clustering algorithm called DI-SIM for asymmetry discovery and directional clustering. A Stochastic co-Blockmodel is introduced to show favorable properties of DI-SIM. To account for the sparse and highly heterogeneous nature of directed networks, DI-SIM uses the regularized graph Laplacian and projects the rows of the eigenvector matrix onto the sphere. A nodewise ASYMMETRY SCORE and DI-SIM are used to analyze the clustering asymmetries in the networks of Enron emails, political blogs, and the Caenorhabditis elegans chemical connectome. In each example, a subset of nodes have clustering asymmetries; these nodes send edges to one cluster, but receive edges from another cluster. Such nodes yield insightful information (e.g., communication bottlenecks) about directed networks, but are missed if the analysis ignores edge direction.C lustering is widely used to study the structure of social, biological, and technological networks because it provides an aggregated and simplified representation of the complex interactions. The difficulty of the clustering problem has inspired an extensive literature devoted to the statistical and computational issues. Spectral approximation algorithms have become popular due to their computational speed and empirical performance across domain areas.In the clustering literature, the vast majority of the models and algorithms presumes that the interactions are symmetric or undirected. In some settings, the relationships can be well approximated as symmetric. However, asymmetric or directed relationships more fully represent the vast majority of interactions. For example, in the gene regulatory network, one gene drives the transcription of the other gene. In the power grid network, electricity flows from one node to the other. In a communication network, one node initiates the conversation. In other examples, it might be easier to observe the relationship without direction, but the direction remains of fundamental importance. For example, in a social network, a business searching for "trend leaders" wants to know the direction of influence in relationships, which is not directly observable. In a regulatory network, knockout experiments seek to estimate the direction of gene regulation. For many questions of interest, making the edges undirected does not provide an appropriate approximation. In all of these examples, the direction of the edges is essential to the function of the network. Directionality gives asymmetry to a relationship and the standard notion of clustering is insufficient to explore and appropriately aggregate asymmetric relationships in our data examples.To extend clustering to directed networks, we use Hartigan's notion of co-clustering, which he proposed as a way to simultaneously cluster both the rows and the columns o...

The Block Point Process Model for Continuous-time Event-based Dynamic Networks

“…Recent work has focused on estimating the class memberships from the adjacency matrix A. In this setting, spectral clustering (and regularized variants) has emerged as an efficient estimator that has theoretical accuracy guarantees [22,31,[33][34][35], scales to large networks with thousands of nodes, and is generally not sensitive to initialization.…”

“…Spectral clustering is an attractive choice because it scales to large networks containing thousands of nodes and has theoretical performance guarantees applicable to the BPPM, as we discuss next. Recent work has demonstrated that applying spectral clustering (or a regularized variant) to a network generated from an SBM results in consistent estimates of class assignments as the number of nodes N → ∞ [22,31,[33][34][35]. These theoretical guarantees typically require the expected degrees of nodes to grow polylogarithmically with the number of nodes so that the network is not too sparse.…”

A Dynamic Network Perspective on the Latent Group Structure of Cryptocurrencies

“…which controls the number of the offspring of node j. To analyze the asymmetric adjacency matrix A t caused by directional information, Rohe et al (2016) propose using the singular value decomposition instead of eigen-decomposition for the regularized graph Laplacian. The intuition behind this methodology is to use both the eigenvectors of L τ,t L τ,t and L τ,t L τ,t , which contains information about "the number of common parents" and "the number of common offspring"; that is, for each t = 1, · · · , T ,…”

“…Although their return patterns are closely related, the fundamental attributes between them are rather different: BTC employs SHA256 while LTC uses Scrypt. As a comparison, we also show the grouping results for the same 20 cryptos under DISIM from Rohe et al (2016) in Table 2.…”

“…CA-DI-SIM-Dym represents Algorithm 2. DI-SIM-DC is the degree-corrected DI-SIM inRohe et al (2016) and CA-DI-SIM-Stc is the static covariate-assisted DI-SIM method inZhang et al (2018). In all cases, the network size is fixed at 100, and the number of membership changes varies in {0, N/50, N/25, N/20, N/10, N/5, N/4, N/2, N }.The horizon is T = 10 and all simulations are repeated 100 times.…”