Selective Cramér-Rao Bound For Estimation After Model Selection

Meir, Elad; Routtenberg, Tirza

doi:10.1109/ssp.2018.8450764

Cited by 4 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This relation can be used for obtaining a lower bound on the MSE of coherent and selective unbiased estimators directly from any CRB-type lower bound on the MSSE. The main contribution of this work, the selective CRB, is presented in Section III-C, followed by important special cases, in Section III-D. An early derivation of the selective CRB for a scalar cost function appears in [55].…”

Section: Selective Crbmentioning

confidence: 99%

Cramer-Rao Bound for Estimation After Model Selection and its Application to Sparse Vector Estimation

Meir¹,

Routtenberg²

2019

Preprint

Self Cite

View full text Add to dashboard Cite

In many practical parameter estimation problems, such as coefficient estimation of polynomial regression and direction-of-arrival (DOA) estimation, model selection is performed prior to estimation. In these cases, it is assumed that the true measurement model belongs to a set of candidate models. The data-based model selection step affects the subsequent estimation, which may result in a biased estimation. In particular, the oracle Cramér-Rao bound (CRB), which assumes knowledge of the model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we analyze the estimation performance of post-modelselection estimators, by using the mean-squared-selected-error (MSSE) criterion. We assume coherent estimators that force unselected parameters to zero, and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cramér-Rao-type bound on the MSSE and on the mean-squared-error (MSE) of any coherent and selective unbiased estimators. As an important special case, we illustrate the computation and applicability of the proposed selective CRB for sparse vector estimation, in which the selection of a model is equivalent to the recovery of the support. Finally, we demonstrate in numerical simulations that the proposed selective CRB is a valid lower bound on the performance of the post-model-selection maximum likelihood estimator for general linear model with different model selection criteria, and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the existing bounds: oracle CRB, averaged CRB, and the SMS-CRB from [1].

show abstract

Section: Selective Crbmentioning

confidence: 99%

Cramer-Rao Bound for Estimation After Model Selection and its Application to Sparse Vector Estimation

Meir¹,

Routtenberg²

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our case, the observation model is assumed to be perfectly known and the selection approach selects the parameter of interest. In contrast, in the derivation of post-model-selection estimation methods [33] such as regression [29], [30], [34]- [36], as well as the associated performance bounds [37]- [39], the observation model is assumed to be unknown and is selected from a pool of candidate models. Our work is also different from [40]- [44], where the useful data has been determined according to the selection rule.…”

Section: Introductionmentioning

confidence: 99%

Low-Complexity Methods for Estimation After Parameter Selection

Harel

Routtenberg

2020

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

Statistical inference of multiple parameters often involves a preliminary parameter selection stage. The selection stage has an impact on subsequent estimation, for example by introducing a selection bias. The post-selection maximum likelihood (PSML) estimator is shown to reduce the selection bias and the post-selection mean-squared-error (PSMSE) compared with conventional estimators, such as the maximum likelihood (ML) estimator. However, the computational complexity of the PSML is usually high due to the multi-dimensional exhaustive search for a global maximum of the post-selection log-likelihood (PSLL) function. Moreover, the PSLL involves the probability of selection that, in general, does not have an analytical form. In this paper, we develop new low-complexity post-selection estimation methods for a two-stage estimation after parameter selection architecture. The methods are based on implementing the iterative maximization by parts (MBP) approach, which is based on the decomposition of the PSLL function into "easilyoptimized" and complicated parts.The proposed second-best PSML method applies the MBP-PSML algorithm with a pairwise probability of selection between the two highest-ranked parameters w.r.t. the selection rule. The proposed SA-PSML method is based on using stochastic approximation (SA) and Monte Carlo integrations to obtain a non-parametric estimation of the gradient of the probability of selection and then applying the MBP-PSML algorithm on this approximation. For low-complexity performance analysis, we develop the empirical post-selection Cramér-Rao-type lower bound. Simulations demonstrate that the proposed post-selection estimation methods are tractable and reduce both the bias and the PSMSE, compared with the ML estimator, while only requiring moderate computational complexity.

show abstract