Optimal post-selection inference for sparse signals: a nonparametric empirical Bayes approach

Abstract

Summary Many recently developed Bayesian methods have focused on sparse signal detection. However, much less work has been done addressing the natural follow-up question: how to make valid inferences for the magnitude of those signals after selection. Ordinary Bayesian credible intervals suffer from selection bias, as do ordinary frequentist confidence intervals. Existing Bayesian approaches for correcting this bias produce credible intervals with poor frequentist properties. Further, existing frequentist approaches require sacrificing the benefits of shrinkage typical in Bayesian methods, resulting in confidence intervals that are needlessly wide. We address this gap by proposing a nonparametric empirical Bayes approach for constructing optimal selectionadjusted confidence sets. Our method produces confidence sets that are as short as possible on average, while both adjusting for selection and maintaining exact frequentist coverage uniformly over the parameter space. We demonstrate an important consistency property of our procedure: under mild conditions, it asymptotically converges to the results of an oracle-Bayes analysis in which the prior distribution of signal sizes is known exactly. Across a series of examples, the method outperforms existing frequentist techniques for post-selection inference, producing confidence sets that are notably shorter but with the same coverage guarantee. All proofs may be found in the Supplementary Material.

Differentially Private Confidence Interval for Extrema of Parameters

“…Rosenkranz (2016) and Stallard et al (2008) have considered some ad-hoc methods to adjust selection bias, but those methods lack theoretical justifications. Bornkamp et al (2017) and Woody et al (2022) consider Bayesian approach, which is clearly model-dependent and often lacks frequentist interpretation. Recently, several bootstrap-based CI for extrema parameter have been proposed in Guo and He (2021) and Guo et al (2022) among others.…”

Subgroup analysis and adaptive experiments crave for debiasing

“…Several attempts have been made to address the subgroup selection bias. Some existing procedures based on a plug‐in correction of the selection bias are not well grounded (Lee & Shen, 2018), Bayesian methods tend to lack frequentist interpretations (Woody et al, 2022), and simultaneous inference‐based approaches tend to be conservative as they aim to control the family‐wise error rate for all candidate subgroups (Dezeure et al, 2017; Hall & Miller, 2010). Those conservative simultaneous inference procedures are statistically valid but can be costly in subgroup analysis in that they may have inadequate power to confirm the most promising or most vulnerable subgroup.…”

Empirical Bayes and Selective Inference

“…We say that θ is random if the joint sampling scheme for the parameter and data is such that the pairs (θ , X) are sampled from their joint distribution until ψ ≡ ψ(θ) gets selected and say that θ is fixed if θ is sampled from its marginal distribution, held fixed, and X sampled from its conditional distribution X|θ until ψ is selected for inference. Woody et al 23 refer to these two scenarios as 'joint selection' and 'conditional selection' , respectively. As above, let R be the binary random variable that indicates if selection of the parameter ψ under consideration has happened.…”