Optimal shrinkage of eigenvalues in the spiked covariance model

Abstract

We show that in a common high-dimensional covariance model, the choice of loss function has a profound effect on optimal estimation. In an asymptotic framework based on the Spiked Covariance model and use of orthogonally invariant estimators, we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker η that acts elementwise on the sample eigenvalues. Indeed, to each loss function there corresponds a unique admissible eigenvalue shrinker η* dominating all other shrinkers. The shape of the optimal shrinker is determined by the choice of loss function and, crucially, by inconsistency of both eigenvalues and eigenvectors of the sample covariance matrix. Details of these phenomena and closed form formulas for the optimal eigenvalue shrinkers are worked out for a menagerie of 26 loss functions for covariance estimation found in the literature, including the Stein, Entropy, Divergence, Fréchet, Bhattacharya/Matusita, Frobenius Norm, Operator Norm, Nuclear Norm and Condition Number losses.

Asymptotics of empirical eigenstructure for high dimensional spiked covariance

“…Therefore, a proper correction or shrinkage is needed. See a recent paper by Donoho, Gavish and Johnstone (2014) for optimal shrinkage of eigenvalues.…”

Estimation of the inverse scatter matrix of an elliptically symmetric distribution

“…Kubokawa and Inoue [25] consider general types of ridge estimators for covariance and precision matrices, and derive asymptotic expansions of their risk functions. More generally, the idea to correct (shrink) the eigenvalues of the sample covariance matrix is also found in previous work by Ledoit and Wolf [29], El Karoui [14], Ledoit and Wolf [30] and Donoho [11]. The problem has been examined under many sparsity scenarios, for example, zero elements of the matrix [2,13,38,6] or its inverse [34,20,37,28,7,36], bandedness [3,4] among others.…”

High‐dimensional covariance matrix estimation

“…With normality of data assumed, under p/n ! c 2 (0, 1] and a spiked covariance model where Σ has r fixed top-eigenvalues followed by all ones, Donoho, Gavish, and Johnstone (2018) derived optimal shrinkers for a wide variety of loss functions, showing that optimality is very loss-function, and hence application, dependent.…”

“…Lam, Feng, and Hu (2017) and Lam and Feng (2018) used similar ideas to construct well-conditioned integrate volatility matrix estimators for intraday and high frequency tick-by-tick data respectively, demonstrating theoretically how the minimum variance portfolio can be benefitted. Donoho, Gavish, and Johnstone (2018) proved that different loss functions can lead to completely different shrinkage formulae for the sample eigenvalues in a spiked covariance model, and worked out such formulae for various loss functions. Engle, Ledoit, and Wolf (2019) proposed to use nonlinearly shrinkage technique to construct a dynamic covariance matrix estimator.…”