High-dimensional covariance matrix estimation with missing observations

Abstract

In this paper, we study the problem of high-dimensional approximately low-rank covariance matrix estimation with missing observations. We propose a simple procedure computationally tractable in highdimension and that does not require imputation of the missing data. We establish non-asymptotic sparsity oracle inequalities for the estimation of the covariance matrix with the Frobenius and spectral norms, valid for any setting of the sample size and the dimension of the observations. We further establish minimax lower bounds showing that our rates are minimax optimal up to a logarithmic factor.

Low separation rank covariance estimation using Kronecker product expansions

“…, where z t ∈ R pq , having zero-mean and covariance equal to (1). A sufficient statistic for covariance estimation is the well-known sample covariance matrix (SCM):…”

“…A penalized least-squares approach was proposed in [1] for estimating a low rank covariance matrix by solving:…”

“…The model (1) with r ≥ 1 has been proposed in spatiotemporal MEG/EEG covariance modeling [23], [24], [25] and SAR data analysis [26]. We finally note that Van Loan and Pitsianis [27] have shown that any pq × pq matrix Σ 0 can be written as an orthogonal expansion of Kronecker products of the form (1).…”

“…First, we propose a novel convex optimization procedure, called the Permuted Rank-Penalized Least Squares (PRLS) method, for estimating covariance matrices with additive KP structure of the form (1 . To the best of our knowledge, this convex approach has not been proposed or studied in the high dimensional covariance estimation problem for estimating matrices of the form (1).…”

“…For covariance matrices of low separation rank, our results establish that PRLS has significantly faster convergence than the standard sample covariance matrix (SCM) estimator. In addition, this framework allows one to tradeoff estimation error for approximation error, thus providing a scalable covariance estimation framework in terms of separation rank, an analog to low rank approximation of covariance matrices [1]. The MSE convergence rates generalize the high dimensional rates recently obtained for the ML Flip-flop algorithm [2], [3].…”