Two-sample hypothesis testing for inhomogeneous random graphs

Ghoshdastidar, Debarghya; Gutzeit, Maurilio; Carpentier, Alexandra; Luxburg, Ulrike von

doi:10.1214/19-aos1884

Cited by 32 publications

(27 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the discrete setting, the L 1 norm is often considered [29], as well as the L 2 norm [6]. The paper [17] considers the two sample testing problem in inhomogeneous random graphs, in order to study the effect of various distances (total variation, Frobenius distance, operator norm, Kullback-Leibler divergence). The paper [9] considers the goodness-of-fit testing problem in inhomogeneous random graphs for the Frobenius and operator norm distances.…”

Section: Extension To L T Distancementioning

confidence: 99%

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

Chhor¹,

Carpentier²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the goodness-of fit testing problem for Hölder smooth densities over R d : given n iid observations with unknown density p and given a known density p 0 , we investigate how large ρ should be to distinguish, with high probability, the case p = p 0 from the composite alternative of all Hölder-smooth densities p such that p − p 0 t ≥ ρ where t ∈ [1, 2]. The densities are assumed to be defined over R d and to have Hölder smoothness parameter α > 0. In the present work, we solve the case α ≤ 1 and handle the case α > 1 using an additional technical restriction on the densities. We identify matching upper and lower bounds on the local minimax rates of testing, given explicitly in terms of p 0 . We propose novel test statistics which we believe could be of independent interest. We also establish the first definition of an explicit cutoff u B allowing us to split R d into a bulk part (defined as the subset of R d where p 0 takes only values greater than or equal to u B ) and a tail part (defined as the complementary of the bulk), each part involving fundamentally different contributions to the local minimax rates of testing.

show abstract

Section: Extension To L T Distancementioning

confidence: 99%

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

Chhor¹,

Carpentier²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Assumption 1 is standard in graphon estimation literature (Klopp et al, 2017) since it avoids graphons corresponding to inhomogeneous random graph models. It is known that two graphs from widely separated inhomogeneous models (in L 2distance) are statistically indistinguishable (Ghoshdastidar et al, 2020), and hence, it is essential to ignore such models to derive meaningful guarantees. Assumption 2 ensures that, under a measurepreserving transformation, the graphon has strictly increasing degree function, which is a canonical representation of an equivalence class of graphons (Bickel & Chen, 2009).…”

Section: Graph Distance Based On Graphonsmentioning

confidence: 99%

“…Two-sample testing is usually studied in the large sample case m → ∞, and several nonparametric tests are known that could also be applied to graphs. However, in the context of graphs, it is relevant to study the small sample setting, particularly m = 2, that is, the problem of deciding if two large graphs are statistically identical or not (Ghoshdastidar et al, 2020;Agterberg et al, 2020).…”

Section: Graph Two-sample Testingmentioning

confidence: 99%

“…Existing works consider alternative random graph model, such as inhomogeneous Erdős-Rényi models or random dot product graph models, which are more restrictive. The condition φ > 0 is necessary if one only has access to finitely many independent samples (Ghoshdastidar et al, 2020). A two-sample test T is a binary function of the given samples such that T = 1 denotes that the test rejects the null hypothesis H 0 and T = 0 implies that the test rejects the alternate hypothesis H a .…”

Section: Graph Two-sample Testingmentioning

confidence: 99%

See 1 more Smart Citation

Graphon based Clustering and Testing of Networks: Algorithms and Theory

Mahalakshmi¹,

Vankadara²,

Ghoshdastidar³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Network-valued data are encountered in a wide range of applications, and pose challenges in learning due to their complex structure and absence of vertex correspondence. Typical examples of such problems include classification or grouping of protein structures and social networks. Various methods, ranging from graph kernels to graph neural networks, have been proposed that achieve some success in graph classification problems. However, most methods have limited theoretical justification, and their applicability beyond classification remains unexplored. In this work, we propose methods for clustering multiple graphs, without vertex correspondence, that are inspired by the recent literature on estimating graphonssymmetric functions corresponding to infinite vertex limit of graphs. We propose a novel graph distance based on sorting-and-smoothing graphon estimators. Using the proposed graph distance, we present two clustering algorithms and show that they achieve state-of-the-art results. We prove the statistical consistency of both algorithms under Lipschitz assumptions on the graph degrees. We further study the applicability of the proposed distance for graph two-sample testing problems.

show abstract

“…To overcome this challenge, Tang et al (2017a) proposed a semi-parametric two-sample test for a class of latent position random graphs, and studied the problem of testing whether two dot product random graphs are drawn from the same population or not. Other testing approaches that focused on hypothesis testing for specific scenarios, such as sparse networks ( Ghoshdastidar et al, 2017a ) and networks with a large number of nodes ( Ghoshdastidar et al, 2017b ), have been developed. More recently, Ghoshdastidar and von Luxburg (2018) developed a novel testing framework for random graphs, particularly for the cases with small sample sizes and the large number of nodes, and studied its optimality.…”

Section: Introductionmentioning

confidence: 99%

Preventing Failures by Dataset Shift Detection in Safety-Critical Graph Applications

Song

Thiagarajan

Kailkhura

2021

Front. Artif. Intell.

View full text Add to dashboard Cite

Dataset shift refers to the problem where the input data distribution may change over time (e.g., between training and test stages). Since this can be a critical bottleneck in several safety-critical applications such as healthcare, drug-discovery, etc., dataset shift detection has become an important research issue in machine learning. Though several existing efforts have focused on image/video data, applications with graph-structured data have not received sufficient attention. Therefore, in this paper, we investigate the problem of detecting shifts in graph structured data through the lens of statistical hypothesis testing. Specifically, we propose a practical two-sample test based approach for shift detection in large-scale graph structured data. Our approach is very flexible in that it is suitable for both undirected and directed graphs, and eliminates the need for equal sample sizes. Using empirical studies, we demonstrate the effectiveness of the proposed test in detecting dataset shifts. We also corroborate these findings using real-world datasets, characterized by directed graphs and a large number of nodes.

show abstract

Two-sample hypothesis testing for inhomogeneous random graphs

Cited by 32 publications

References 31 publications

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

Graphon based Clustering and Testing of Networks: Algorithms and Theory

Preventing Failures by Dataset Shift Detection in Safety-Critical Graph Applications

Contact Info

Product

Resources

About