Inference of hyperedges and overlapping communities in hypergraphs

Abstract

Hypergraphs, encoding structured interactions among any number of system units, have recently proven a successful tool to describe many real-world biological and social networks. Here we propose a framework based on statistical inference to characterize the structural organization of hypergraphs. The method allows to infer missing hyperedges of any size in a principled way, and to jointly detect overlapping communities in presence of higher-order interactions. Furthermore, our model has an efficient numerical implementation, and it runs faster than dyadic algorithms on pairwise records projected from higher-order data. We apply our method to a variety of real-world systems, showing strong performance in hyperedge prediction tasks, detecting communities well aligned with the information carried by interactions, and robustness against addition of noisy hyperedges. Our approach illustrates the fundamental advantages of a hypergraph probabilistic model when modeling relational systems with higher-order interactions.

Generalized Inference of Mesoscale Structures in Higher-order Networks

“…Another advantage of our inference procedure is that it is stable and reliable for extremely large hyperedges. Due to computational and numerical constraints, previous models were also limited to consider hyperedges with maximal size D = 25 (35,50). As we illustrate in Section VII with an Amazon and a Gene-Disease dataset, large interactions (respectively D = 9350 and D = 1074) should not be neglected as they provide useful information and significantly boost the quality of inference…”

“…In the last few years several tools have been developed for higher-order network analysis. These include higherorder centrality scores (28,29), clustering (30) and motif analysis (31,32), as well as higher-order approaches to network backboning (33,34), link prediction (35), and methods to reconstruct non-dyadic relationships from pairwise interaction records (36). A variety of approaches have been suggested to detect modules in hypergraphs, including nonparametric methods with hypergraphons (37), tensor decompositions (38), latent space distance models (39), latent class models (40), flow-based algorithms (41,42), spectral clustering (43)(44)(45) and spectral embeddings (46).…”

“…Recently, statistical inference frameworks have been proposed to capture in a principled way the mesoscale organization of higher-order networks (35,50,51). Despite their success, current approaches suffer from a number of significant drawbacks, as they are restricted to hyperedges that only contain a limited number of nodes (51), which often results in discarding a portion of the available list of hyperedges.…”

“…Furthermore, they can be computationally slow, as their complexity is often more than linear in the system size (50), making them unsuitable for the study of large-scale real-world systems. Finally, they are typically constrained to work in specific cases only, for instance assuming assortative community structure (35).…”

“…The term n(n − 1)/2 averages among the number of possible pairwise interaction among the n nodes in the hyperedge. Note that previous generative models for hypergraphs were limited to detect only assortative community interactions (35,50). By contrast, in our model each entry w kq distinctly specifies the strength of the interactions between each k, q community pair.…”