Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures

Abstract

The natural habitat of most Bayesian methods is data represented by exchangeable sequences of observations, for which de Finetti's theorem provides the theoretical foundation. Dirichlet process clustering, Gaussian process regression, and many other parametric and nonparametric Bayesian models fall within the remit of this framework; many problems arising in modern data analysis do not. This article provides an introduction to Bayesian models of graphs, matrices, and other data that can be modeled by random structures. We describe results in probability theory that generalize de Finetti's theorem to such data and discuss their relevance to nonparametric Bayesian modeling. With the basic ideas in place, we survey example models available in the literature; applications of such models include collaborative filtering, link prediction, and graph and network analysis. We also highlight connections to recent developments in graph theory and probability, and sketch the more general mathematical foundation of Bayesian methods for other types of data beyond sequences and arrays.

Bootstrap quantification of estimation uncertainties in network degree distributions

“…We denote the probability that a randomly selected node has a degree k by f ( k ), the degree distribution of G by , and the mean degree of G by μ ( G ). We assume that G is involution invariant 23, 24 , that is from the vantage point of any randomly selected vertex, the rest of the connected network is probabilistically the same.…”

Random-Walk Models of Network Formation and Sequential Monte Carlo Methods for Graphs

“…If the missing structure is relevant to statistical analysis, its absence constitutes a form of model misspecification. To put the problem into context, it is useful to compare some commonly used models: (a)graphon models (as mentioned above) (these have recently received considerable attention in statistics and include stochastic block models (Goldenberg et al ., ), and many models that are popular in machine learning (see Orbanz and Roy ()); (b)models that randomize a graph to match a given degree sequence, such as the configuration model (Durrett, ); (c)models describing a formation process, such as PA models (Barabási and Albert, ). …”

“…Borgs et al . () and Orbanz and Roy ()): the generated graph is dense; unless it is k ‐partite or disconnected, the distance between any two vertices in an infinite graph is almost surely 1 or 2, etc. A graphon can encode any fixed pattern on some number n of vertices, but this pattern then occurs on every possible subgraph of size n with fixed probability. (b)Configuration models are popular in probability (because of their simplicity) but have limited use in statistics unless the quantity of interest is the degree sequence itself: they are ‘maximally random’ given the degrees, in a manner similar to exponential families being maximally random given a sufficient statistic, and are thus insensitive to any structure that is not captured by the degrees.…”

“…(a) graphon models (as mentioned above) (these have recently received considerable attention in statistics and include stochastic block models (Goldenberg et al, 2010), and many models that are popular in machine learning (see Orbanz and Roy (2015)); (b) models that randomize a graph to match a given degree sequence, such as the configuration model (Durrett, 2006); (c) models describing a formation process, such as PA models (Barabási and Albert, 1999).…”

“…(a) In the case of graphon models, various symptoms of this problem have been noted (e.g. Borgs et al (2014) and Orbanz and Roy (2015)): the generated graph is dense; unless it is k-partite or disconnected, the distance between any two vertices in an infinite graph is almost surely 1 or 2, etc. A graphon can encode any fixed pattern on some number n of vertices, but this pattern then occurs on every possible subgraph of size n with fixed probability.…”