Regularized estimation of large covariance matrices

Bickel, Peter J.; Levina, Elizaveta

doi:10.1214/009053607000000758

Cited by 1,065 publications

(1,213 citation statements)

References 28 publications

Supporting

Mentioning

1,199

Contrasting

Unclassified

Order By: Relevance

“…The estimated covariance matrix in each case is close to the true value, with small mean square error, average and maximum absolute bias. We compare the estimation of the covariance matrix to a recent method by Bickel & Levina (2008) which bands the sample covariance matrix and proposes a resampling scheme for choosing the optimal banding parameter. The stochastic EM algorithm was also used to arrive at an approximate maximum a posteriori estimate of the covariance matrix.…”

Section: ·1 Factor Selection and Covariance Matrix Estimationmentioning

confidence: 99%

Sparse Bayesian infinite factor models

Bhattacharya¹,

Dunson²

2011

Biometrika

334

480

View full text Add to dashboard Cite

SUMMARYWe focus on sparse modelling of high-dimensional covariance matrices using Bayesian latent factor models. We propose a multiplicative gamma process shrinkage prior on the factor loadings which allows introduction of infinitely many factors, with the loadings increasingly shrunk towards zero as the column index increases. We use our prior on a parameter-expanded loading matrix to avoid the order dependence typical in factor analysis models and develop an efficient Gibbs sampler that scales well as data dimensionality increases. The gain in efficiency is achieved by the joint conjugacy property of the proposed prior, which allows block updating of the loadings matrix. We propose an adaptive Gibbs sampler for automatically truncating the infinite loading matrix through selection of the number of important factors. Theoretical results are provided on the support of the prior and truncation approximation bounds. A fast algorithm is proposed to produce approximate Bayes estimates. Latent factor regression methods are developed for prediction and variable selection in applications with high-dimensional correlated predictors. Operating characteristics are assessed through simulation studies, and the approach is applied to predict survival times from gene expression data.

show abstract

Section: ·1 Factor Selection and Covariance Matrix Estimationmentioning

confidence: 99%

Sparse Bayesian infinite factor models

Bhattacharya¹,

Dunson²

2011

Biometrika

334

480

View full text Add to dashboard Cite

show abstract

“…This type of structure has also been used for estimating a high-dimensional covariance matrix (Bickel and Levina, 2008). In addition, because the correlations among variables (i.e.…”

Section: C2 Given a Set Of Multivariate Random Vectorsmentioning

confidence: 99%

“…In practice, Σ has to be estimated. We may apply an existing method, such as the banding and thresholding technique, to estimate a high-dimensional sparse covariance matrix (Bickel and Levina, 2008;Cai and Liu, 2011); see Cai et al (2016) for an excellent review. Under the α mixing assumption C2, σ ij is close to zero when |i − j| is large and thus we may apply the banding approach of Bickel and Levina (2008) …”

Section: C9 the Conditionally α-Mixingmentioning

confidence: 99%

“…Theoretically optimal bandwidth k n and minimax risk rates ofΣ kn have been studied in Bickel and Levina (2008). Since a theoretically optimal k n is determined by some unknown hyper-parameters, we use five-fold cross-validation to select an optimal bandwidth k n (Bickel and Levina, 2008;Cai and Liu, 2011).…”

Section: C9 the Conditionally α-Mixingmentioning

confidence: 99%

“…Since a theoretically optimal k n is determined by some unknown hyper-parameters, we use five-fold cross-validation to select an optimal bandwidth k n (Bickel and Levina, 2008;Cai and Liu, 2011). Following Xu et al (2016), under the assumptions in Theorem 2, we can show thatμ(γ) andσ 2 (γ) estimated based on the bandable covariance matrixΣ kn satisfyμ(γ) = {1+o(1)}µ(γ)…”

Section: C9 the Conditionally α-Mixingmentioning

confidence: 99%

See 2 more Smart Citations

An Adaptive Test on High-dimensional Parameters in Generalized Linear Models

Pan³

2019

STAT SINICA

View full text Add to dashboard Cite

Significance testing for high-dimensional generalized linear models (GLMs) has been increasingly needed in various applications, however, existing methods are mainly based on a sum of squares of the score vector and only powerful under certain alternative hypotheses. In practice, depending on whether the true association pattern under an alternative hypothesis is sparse or dense or between, the existing tests may or may not be powerful. In this paper, we propose an adaptive test on a high-dimensional parameter of a GLM (in the presence of a low-dimensional nuisance parameter), which can maintain high power across a wide range of scenarios. To evaluate its p-value, its asymptotic null distribution is derived. We conduct simulations to demonstrate the superior performance of the proposed test. In addition, we apply it and other existing tests to an Alzheimer's Disease Neuroimaging Initiative (ADNI) * Correspondence: wuxx0845@umn.edu (C.W.), weip@biostat.umn.edu (W.P.) † Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at:

show abstract

E nsemble M odels

Fuentes¹,

Foley²

2012

Encyclopedia of Environmetrics

View full text Add to dashboard Cite

An ensemble, or sample, of competing numerical models has been used in many applications to represent different predictions of the true state of a physical system. Ensembles of computer models (e.g. weather, climate, air quality, ocean models) are often used to forecast future states of a physical system and to quantify uncertainty in the numerical model predictions. Various statistical methods have been proposed to improve ensemble predictions from deterministic computer simulations based on actual measurements of the physical systems (e.g. data assimilation, Bayesian model averaging). Ensemble data mining methods have also been developed for a wide variety of applications to combine different versions of a statistical model (e.g. time series models, simple regression models, neural networks) to improve the predictive model performance. We present different statistical criteria that have been proposed to select or weight ensemble members for both numerical model‐based and statistical model‐based ensembles.

show abstract

Regularized estimation of large covariance matrices

Cited by 1,065 publications

References 28 publications

Sparse Bayesian infinite factor models

Sparse Bayesian infinite factor models

An Adaptive Test on High-dimensional Parameters in Generalized Linear Models

E nsemble M odels

Contact Info

Product

Resources

About