Universal Weighted MSE Improvement of the Least-Squares Estimator

Abstract

Abstract-Since the seminal work of Stein in the 1950s, there has been continuing research devoted to improving the total meansquared error (MSE) of the least-squares (LS) estimator in the linear regression model. However, a drawback of these methods is that although they improve the total MSE, they do so at the expense of increasing the MSE of some of the individual signal components. Here we consider a framework for developing linear estimators that outperform the LS strategy over bounded norm signals, under all weighted MSE measures. This guarantees, for example, that both the total MSE and the MSE of each of the elements will be smaller than that resulting from the LS approach. We begin by deriving an easily verifiable condition on a linear method that ensures LS domination for every weighted MSE. We then suggest a minimax estimator that minimizes the worst-case MSE over all weighting matrices and bounded norm signals subject to the universal weighted MSE domination constraint.

Rethinking Biased Estimation: Improving Maximum Likelihood and the Cramér–Rao Bound

“…A similar expression can be obtained for the weighted MSE [48]. Ideally, to obtain the tightest possible MSE bound, we would like to minimize .2) is equal to 0 and is achieved at θ 0 by the estimateθ = θ 0 which clearly cannot be implemented.…”

“…Although in our derivations we treat the (unweighted) MSE, the essential ideas introduced in this section can be generalized to include weighted MSE criteria which measure the average weighted squarednorm error [48].…”

“…In an estimation context, where our prime concern is inferring θ 0 , the MSE (or weighted MSE) provides a direct measure of the relevant performance. Although herein we focus on the MSE, the essential ideas can be easily generalized to include weighted MSE criteria which measure the average weighted squarednorm error [48].…”

Three-dimensional imaging algorithm for tomography SAR based on multiple signal classification

“…The reflectivities of the targets should be estimated by the amplitude estimator. The cost function of least squares estimator can be written as [8] corresponding to the received data of n th look, which can be obtained by minimizing the above cost function (8)…”

MMSE estimation in a linear signal model with ellipsoidal constraints

“…The first case has attracted a lot of interest in the past, see e.g. [1][2][3][4][5] where the linear minimax estimator is sought which minimizes the worst case mean squared error. It is well known that the ordinary least squares estimator is outperformed by this estimator.…”