Covariance, subspace, and intrinsic Crame/spl acute/r-Rao bounds

Abstract

Cramér-Rao bounds on estimation accuracy are established for estimation problems on arbitrary manifolds in which no set of intrinsic coordinates exists. The frequently encountered examples of estimating either an unknown subspace or a covariance matrix are examined in detail. The set of subspaces, called the Grassmann manifold, and the set of covariance (positive-definite Hermitian) matrices have no fixed coordinate system associated with them and do not possess a vector space structure, both of which are required for deriving classical Cramér-Rao bounds. Intrinsic versions of the Cramér-Rao bound on manifolds utilizing an arbitrary affine connection with arbitrary geodesics are derived for both biased and unbiased estimators. In the example of covariance matrix estimation, closed-form expressions for both the intrinsic and flat bounds are derived and compared with the root-mean-square error (RMSE) of the sample covariance matrix (SCM) estimator for varying sample support. The accuracy bound on unbiased covariance matrix estimators is shown to be about (10 log 10) 1 2 dB, where is the matrix order. Remarkably, it is shown that from an intrinsic perspective, the SCM is a biased and inefficient estimator and that the bias term reveals the dependency of estimation accuracy on sample support observed in theory and practice. The RMSE of the standard method of estimating subspaces using the singular value decomposition (SVD) is compared with the intrinsic subspace Cramér-Rao bound derived in closed form by varying both the signal-to-noise ratio (SNR) of the unknown-dimensional subspace and the sample support. In the simplest case, the Cramér-Rao bound on subspace estimation accuracy is shown to be about (()) 1 2 1 2 SNR 1 2 rad for-dimensional subspaces. It is seen that the SVD-based method yields accuracies very close to the Cramér-Rao bound, establishing that the principal invariant subspace of a random sample provides an excellent estimator of an unknown subspace. The analysis approach developed is directly applicable to many other estimation problems on manifolds encountered in signal processing and elsewhere, such as estimating rotation matrices in computer vision and estimating subspace basis vectors in blind source separation.

A Note on the Intrinsic Cramer-Rao Bound

“…It consists of some remarks on intrinsic Cramer-Rao bounds, and can be viewed as an introduction to the paper [6]. In particular, we show that the results contained in [6] allow to derive in a straightforward way the fact that intrinsic RMSE can always be expected not to depend on the underlying parameter. This is a worthy to note result, which is not explicitely stated in [6].…”

“…[3] and references therein), and more generally information geometry [1] allows to develop Cramer-Rao bounds that quantify the goodness of an estimator through intrinsic tools. For more information on Cramer-Rao analysis on manifolds see also [6] and the long list of references therein.…”

“…Smith in [6]. In this latter paper, the author proposes a derivation of the intrinsic CRLB, and presents the results in a way that is relevant to the field of signal processing.…”

Horoball Hulls and Extents in Positive Definite Space

“…It is important to note that in all these examples, the Riemannian structure of P(n) is a crucial element of the modelling process: merely treating the matrices as points in R n 2 and endowing the space with the Euclidean distance 1 does not capture the correct notion of distance or closeness that is meaningful in these applications [22,21,27,25,23]. For example, if we want to interpolate between two matrices of similar volume (determinant), a line between the two in R n 2 contains matrices of volume well outside the range of volumes of the two matrices, but all matrices on a Riemannian geodesic have volume in the range.…”

A Fresh Look at the Bayesian Bounds of the Weiss-Weinstein Family

“…Consequently, they are used as a benchmark in order to evaluate the performance of an estimator and to determine if an improvement is possible. The Cramér-Rao bound [3]- [8] has been the most widely used by the signal processing community and is still under investigation from a theoretical point of view (particularly throughout the differential variety in the Riemannian geometry framework [9]- [14]) as from a practical point of view (see, e.g., [15]- [19]). But the Cramér-Rao bound suffers from some drawbacks when the scenario becomes critical.…”