Information-theoretic limits of graphical model selection in high dimensions

Santhanam, Narayana; Wainwright, Martin J.

doi:10.1109/isit.2008.4595367

Cited by 10 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For bounded degree graphs, our results show that the structure can be recovered with high probability once n/ log(p) is sufficiently large. Up to constant factors, this result matches known information-theoretic lower bounds (Bresler et al, 2008, Santhanam andWainwright, 2008). On the other hand, our experimental results on graphs with growing degrees (star-shaped graphs) are consistent with the conjecture that the logistic regression procedure exhibits a threshold at a sample size n = Θ(d log p), at least for problems where the minimum value θ * min stays bounded away from zero.…”

Section: Discussionsupporting

confidence: 90%

“…However, in the absence of additional restrictions, the computational complexity of the method is O(p d+1 ). Santhanam and Wainwright (2008) analyze the information-theoretic limits of graphical model selection, providing both upper and lower bounds on various model selection procedures, but these methods also have prohibitive computational cost.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High-Dimensional Graphical Model Selection Using l1 -Regularized Logistic Regression

Wainwright¹,

Ravikumar²,

Lafferty³

2007

Advances in Neural Information Processing Systems 19

View full text Add to dashboard Cite

We consider the problem of estimating the graph structure associated with a discrete Markov random field. We describe a method based on ℓ 1 -regularized logistic regression, in which the neighborhood of any given node is estimated by performing logistic regression subject to an ℓ 1 -constraint. Our framework applies to the high-dimensional setting, in which both the number of nodes p and maximum neighborhood sizes d are allowed to grow as a function of the number of observations n. Our main results provide sufficient conditions on the triple (n, p, d) for the method to succeed in consistently estimating the neighborhood of every node in the graph simultaneously. Under certain assumptions on the population Fisher information matrix, we prove that consistent neighborhood selection can be obtained for sample sizes n = Ω(d 3 log p), with the error decaying as O(exp(−Cn/d 3 )) for some constant C. If these same assumptions are imposed directly on the sample matrices, we show that n = Ω(d 2 log p) samples are sufficient.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Introductionmentioning

confidence: 99%

High-Dimensional Graphical Model Selection Using l1 -Regularized Logistic Regression

Wainwright¹,

Ravikumar²,

Lafferty³

2007

Advances in Neural Information Processing Systems 19

View full text Add to dashboard Cite

show abstract

“…Erdős-Rényi random graphs. Necessary conditions for graph estimation have been previously characterized for degree-bounded graph ensembles G Deg (p, ∆) [43]. However, these conditions are too loose to be useful for the ensemble of Erdős-Rényi graphs G ER (p, c/p), where the average degree 17 (c) is much smaller than the maximum degree.…”

Section: 1mentioning

confidence: 99%

“…Necessary conditions on structure learning provide lower bounds on the sample complexity for structure learning and have been studied in [38,43,50]. However, a standard assumption that these works make is that the underlying set of graphs is uniformly distributed with bounded degree.…”

Section: Anandkumar Tan Huang and Willskymentioning

confidence: 99%

High-dimensional structure estimation in Ising models: Local separation criterion

Anandkumar¹,

Tan²,

Huang³

et al. 2012

Ann. Statist.

108

165

View full text Add to dashboard Cite

We consider the problem of high-dimensional Ising (graphical) model selection. We propose a simple algorithm for structure estimation based on the thresholding of the empirical conditional variation distances. We introduce a novel criterion for tractable graph families, where this method is efficient, based on the presence of sparse local separators between node pairs in the underlying graph. For such graphs, the proposed algorithm has a sample complexity of n = Ω(J −2 min log p), where p is the number of variables, and Jmin is the minimum (absolute) edge potential in the model. We also establish nonasymptotic necessary and sufficient conditions for structure estimation.

show abstract

“…Subsequent to our work being posted on the Arxiv, Santhanam and Wainwright [4] again considered essentially the problem for the Ising model, producing nearly matching lower and upper bounds on the asymptotic sampling complexity. Again their conditions do not apply to the low temperature regime.…”

Section: Related Workmentioning

confidence: 99%

Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms

Bresler

Mossel

Sly

Lecture Notes in Computer Science

177

View full text Add to dashboard Cite

Markov random fields are used to model high dimensional distributions in a number of applied areas. Much recent interest has been devoted to the reconstruction of the dependency structure from independent samples from the Markov random fields. We analyze a simple algorithm for reconstructing the underlying graph defining a Markov random field on n nodes and maximum degree d given observations. We show that under mild non-degeneracy conditions it reconstructs the generating graph with high probability using Θ(dǫ −2 δ −4 log n) samples where ǫ, δ depend on the local interactions. For most local interaction ǫ, δ are of order exp(−O(d)).Our results are optimal as a function of n up to a multiplicative constant depending on d and the strength of the local interactions. Our results seem to be the first results for general models that guarantee that the generating model is reconstructed. Furthermore, we provide explicit O(n d+2 ǫ −2 δ −4 log n) running time bound. In cases where the measure on the graph has correlation decay, the running time is O(n 2 log n) for all fixed d. We also discuss the effect of observing noisy samples and show that as long as the noise level is low, our algorithm is effective. On the other hand, we construct an example where large noise implies non-identifiability even for generic noise and interactions. Finally, we briefly show that in some simple cases, models with hidden nodes can also be recovered.

show abstract

Information-theoretic limits of graphical model selection in high dimensions

Cited by 10 publications

References 23 publications

High-Dimensional Graphical Model Selection Using l1 -Regularized Logistic Regression

High-Dimensional Graphical Model Selection Using l1 -Regularized Logistic Regression

High-dimensional structure estimation in Ising models: Local separation criterion

Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms

Contact Info

Product

Resources

About