Hierarchical structure and the prediction of missing links in networks

Abstract

Networks have in recent years emerged as an invaluable tool for describing and quantifying complex systems in many branches of science. Recent studies suggest that networks often exhibit hierarchical organization, in which vertices divide into groups that further subdivide into groups of groups, and so forth over multiple scales. In many cases the groups are found to correspond to known functional units, such as ecological niches in food webs, modules in biochemical networks (protein interaction networks, metabolic networks or genetic regulatory networks) or communities in social networks. Here we present a general technique for inferring hierarchical structure from network data and show that the existence of hierarchy can simultaneously explain and quantitatively reproduce many commonly observed topological properties of networks, such as right-skewed degree distributions, high clustering coefficients and short path lengths. We further show that knowledge of hierarchical structure can be used to predict missing connections in partly known networks with high accuracy, and for more general network structures than competing techniques. Taken together, our results suggest that hierarchy is a central organizing principle of complex networks, capable of offering insight into many network phenomena.

Link prediction in complex networks: A survey

“…Liu and Lü [73] systematically compared these two indices, LRW and SRW, with five other indices, including three local (or quasi-local) indices, CN, RA and LP, and two other random-walk-based global indices, ACT and RWR, as well as the hierarchical structure method (HSM) proposed by Clauset, Moore and Newman [75] (see Section 4.1 for the detailed introduction of HSM). Table 3 Comparison of algorithms' accuracy quantified by AUC and Precision.…”

“…2, the hierarchical structure of a network can be represented by a dendrogram with N leaves (corresponding to the nodes of the network) and N − 1 internal nodes. Clauset, Moore and Newman [75] introduced a simple model where each internal node r is associated with a probability p r and the connecting probability of a pair of nodes (leaves) is equal to p r ′ where r ′ is the lowest common ancestor of these two nodes. Given a real network G and a dendrogram D, let E r be the number of edges in G whose endpoints have r as their lowest common ancestor in D, and let L r and R r , respectively, …”

“…As Redner said [77], focusing on the hierarchical structure inherent in social and biological networks might provide a smart way to find missing links. Clauset, Moore and Newman [75] proposed a general technique to infer the hierarchical organization from network data and further applied it to predict the missing links.…”

Information filtering via weighted heat conduction algorithm

“…All of us have known the feeling of being overwhelmed by the number of new books, journal articles, and conference proceedings coming out each year [1][2][3][4][5]. Technology has dramatically reduced the barriers to publishing and distributing information.…”

Hierarchical kernel spectral clustering

“…In some applications, a more informative hierarchical representation of the data is desirable (Clauset et al, 2008). This is typically the case when the data contain different natural groupings depending of the scale.…”