Shrinkage estimators of large covariance matrices with Toeplitz targets in array signal processing

Abstract

The problem of estimating a large covariance matrix arises in various statistical applications. This paper develops new covariance matrix estimators based on shrinkage regularization. Individually, we consider two kinds of Toeplitz-structured target matrices as the data come from the complex Gaussian distribution. We derive the optimal tuning parameter under the mean squared error criterion in closed form by discovering the mathematical properties of the two target matrices. We get some vital moment properties of the complex Wishart distribution, then simplify the optimal tuning parameter. By unbiasedly estimating the unknown scalar quantities involved in the optimal tuning parameter, we propose two shrinkage estimators available in the large-dimensional setting. For verifying the performance of the proposed covariance matrix estimators, we provide some numerical simulations and applications to array signal processing compared to some existing estimators.