Testing for high‐dimensional geometry in random graphs

Abstract

We study the problem of detecting the presence of an underlying high‐dimensional geometric structure in a random graph. Under the null hypothesis, the observed graph is a realization of an Erdős‐Rényi random graph G(n, p). Under the alternative, the graph is generated from the G(n,p,d) model, where each vertex corresponds to a latent independent random vector uniformly distributed on the sphere double-struckSd−1, and two vertices are connected if the corresponding latent vectors are close enough. In the dense regime (i.e., p is a constant), we propose a near‐optimal and computationally efficient testing procedure based on a new quantity which we call signed triangles. The proof of the detection lower bound is based on a new bound on the total variation distance between a Wishart matrix and an appropriately normalized GOE matrix. In the sparse regime, we make a conjecture for the optimal detection boundary. We conclude the paper with some preliminary steps on the problem of estimating the dimension in G(n,p,d). © 2016 Wiley Periodicals, Inc. Random Struct. Alg., 49, 503–532, 2016

A Smooth Transition from Wishart to GOE

“…If d is large enough compared to n, then the Wishart matrix becomes approximately like the GOE. Recent work of Bubeck, Ding, Eldan, and Racz [3], and independently Jiang and Li [9], shows that the transition happens when d = Θ n 3 . Specifically, they proved the following theorem, where we write TV for total variation distance.…”

“…It is well known that an n × n Wishart matrix with d degrees of freedom is close to the appropriately centered and scaled Gaussian Orthogonal Ensemble (GOE) if d is large enough (see, e.g., [5]). Recent work [3,9] shows that the transition happens when d = Θ n 3 and in this paper we study this critical window. In Theorem 1.2 below we explicitly compute the total variation distance between the Wishart and GOE matrices when d/n 3 → c ∈ (0, ∞), showing, in particular, that the phase transition from Wishart to GOE is smooth.…”

Permutation Tests for Infection Graphs

“…Turning to the stochastic spreading model, we now show that 1 arises as the first-order coefficient in the series expansion of the likelihood with respect to . This is somewhat reminiscent of the use of signed triangles in the random graph testing literature [9,5,4], which appears in the latter two cases as a first-order approximation to the asymptotic distribution of the log-likelihood. Furthermore, we will see that when considering the likelihood ratio of a test with a specific type of composite null hypothesis against the simple alternative G 1 = { 1 }, the edges-within statistic 1 also appears as the first-order coefficient of the likelihood ratio.…”

Basic models and questions in statistical network analysis

“…However, the result above is not tight, and we seek to understand the fundamental limits to detecting underlying geometry. The dimension threshold for dense graphs was recently found in [14], and it turns out that it is d ≈ n 3 , in the following sense.…”

“…The upside of this is that both of these random matrix ensembles have explicit densities that allow for explicit computation. We explain this connection here in the special case of p = 1/2 for simplicity; see [14] for the case of general p.…”