Derandomizing Knockoffs

Ren, Zhimei; Wei, Yuting; Candès, Emmanuel J.

doi:10.1080/01621459.2021.1962720

Cited by 17 publications

(28 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method shows desirable performance in practice, retaining the power of the original method while reducing variability. Additionally, derandomized knockoffs achieves an empirical FDR that is lower than the target level in our experiments; as an open question for future work, it would be interesting to see under what mild conditions can we derive a sharper FDR bound (for example, as in Meinshausen and Bühlmann (2010); Shah and Samworth (2013) for stability selection, or as in Ren et al (2021) for the previously proposed version of derandomized knockoffs). Many variants and extensions of the knockoffs method have appeared in the literature, including the "fixed-X" setting (Barber and Candès, 2015), robustness results for the model-X setting when P X is not known exactly (Barber et al, 2020), a multienvironment knockoff filter (Li et al, 2021), and knockoffs that incorporate side information (Ren and Candès, 2020).…”

Section: Discussionmentioning

confidence: 93%

“…Previously, many attempts have been made to derandomize knockoffs. As mentioned earlier, Ren et al (2021) propose a derandomizing scheme controlling the expected number of false discoveries; Nguyen et al (2020) introduce a aggregation method aiming at FDR control, but under strong assumptions such as that the null feature importance statistics are i.i.d. In a parallel line of work, Emery and Keich (2019); Gimenez and Zou (2019) consider constructing multiple simultaneous knockoffs to improve the stability of knockoffs-this is in contrast to our attempt to aggregate multiple independent knockoff copies.…”

Section: Additional Related Workmentioning

confidence: 99%

“…As discussed earlier, Ren et al (2021) propose a scheme for derandomizing the knockoffs by thresholding the (unweighted) selection frequency. The algorithm proposed there is designed for the purpose of per familywise error (PFER) control; it is nevertheless of interest to compare the two methods empirically.…”

Section: Thenmentioning

confidence: 99%

“…Empirically, it has been observed that the output can be highly variable from one run to another, which is potentially an undesirable property. To address this, Ren et al (2021) propose the "derandomized knockoffs". After running knockoffs M times on the given dataset, their procedure computes…”

Section: Introductionmentioning

confidence: 99%

“…The final selected set is then given by S kn-derand = {j : Π j ≥ η}, all features X j exceeding some threshold probability of selection for a random run of knockoffs. For this method, Ren et al (2021) establish a guaranteed bound on the expected number of false discoveries, but it appears impossible to prove a bound directly on the FDR.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Derandomized knockoffs: leveraging e-values for false discovery rate control

Ren¹,

Barber²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Model-X knockoffs is a flexible wrapper method for high-dimensional regression algorithms, which provides guaranteed control of the false discovery rate (FDR). Due to the randomness inherent to the method, different runs of model-X knockoffs on the same dataset often result in different sets of selected variables, which is undesirable in practice. In this paper, we introduce a methodology for derandomizing model-X knockoffs with provable FDR control. The key insight of our proposed method lies in the discovery that the knockoffs procedure is in essence an e-BH procedure. We make use of this connection, and derandomize model-X knockoffs by aggregating the e-values resulting from multiple knockoff realizations. We prove that the derandomized procedure controls the FDR at the desired level, without any additional conditions (in contrast, previously proposed methods for derandomization are not able to guarantee FDR control). The proposed method is evaluated with numerical experiments, where we find that the derandomized procedure achieves comparable power and dramatically decreased selection variability when compared with model-X knockoffs.

show abstract

Section: Discussionmentioning

confidence: 93%

Section: Additional Related Workmentioning

confidence: 99%

Section: Thenmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Derandomized knockoffs: leveraging e-values for false discovery rate control

Ren¹,

Barber²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

All that Glitters Is not Gold: Type‐I Error Controlled Variable Selection from Clinical Trial Data

Zimmermann,

Baillie,

Kormaksson

et al. 2024

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Clinical trials are primarily conducted to estimate causal effects, but the data collected can also be invaluable for additional research, such as identifying prognostic measures of disease or biomarkers that predict treatment efficacy. However, these exploratory settings are prone to false discoveries (type‐I errors) due to the multiple comparisons they entail. Unfortunately, many methods fail to address this issue, in part because the algorithms used are generally designed to optimize predictions and often only provide the measures used for variable selection, such as machine learning model importance scores, as a byproduct. To address the resulting unclear uncertainty in the selection sets, the knockoff framework offers a model‐agnostic, robust approach to variable selection with guaranteed type‐I error control. Here, we review the knockoff framework in the setting of clinical data, highlighting main considerations using simulation studies. We also extend the framework by introducing a novel knockoff generation method that addresses two main limitations of previously suggested methods relevant for clinical development settings. With this new method, we empirically obtain tighter bounds on type‐I error control and gain an order of magnitude in computational efficiency in mixed data settings. We demonstrate comparable selections to those of the competing method for identifying prognostic biomarkers for C‐reactive protein levels in patients with psoriatic arthritis in four clinical trials. Our work increases access to the knockoff framework for variable selection from clinical trial data. Hereby, this paper helps to address the current replicability crisis which can result in unnecessary research efforts, increased patient burden, and avoidable costs.

show abstract

Revisiting feature selection for linear models with FDR and power guarantees

Yuan

Feng

2022

J. Korean Stat. Soc.

View full text Add to dashboard Cite

Derandomizing Knockoffs

Cited by 17 publications

References 48 publications

Derandomized knockoffs: leveraging e-values for false discovery rate control

Derandomized knockoffs: leveraging e-values for false discovery rate control

All that Glitters Is not Gold: Type‐I Error Controlled Variable Selection from Clinical Trial Data

Revisiting feature selection for linear models with FDR and power guarantees

Contact Info

Product

Resources

About