Controlling False Discovery Rate Using Gaussian Mirrors

Xing, Xin; Zhao, Zhigen; Liu, Jun S.

doi:10.1080/01621459.2021.1923510

Cited by 21 publications

(17 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, we note that two previous papers (Xing et al, 2019;Dai et al, 2020) have observed that in the equicorrelated case, knockoffs have surprisingly low power when compared to other feature selection methods. Since these papers used the SDP formulation, in Appendix F.3, we exactly replicate these simulations but use MRC knockoffs instead.…”

Section: Simulations On Equicorrelated Designsmentioning

confidence: 67%

See 1 more Smart Citation

Powerful Knockoffs via Minimizing Reconstructability

Asher¹,

Janson²

2020

Preprint

View full text Add to dashboard Cite

Model-X knockoffs (Candès et al., 2018) allows analysts to perform feature selection using almost any machine learning algorithm while still provably controlling the expected proportion of false discoveries. To apply model-X knockoffs, one must construct synthetic variables, called knockoffs, which effectively act as controls during feature selection. The gold standard for constructing knockoffs has been to minimize the mean absolute correlation (MAC) between features and their knockoffs, but, surprisingly, we prove this procedure can be powerless in extremely easy settings, including Gaussian linear models with correlated exchangeable features. The key problem is that minimizing the MAC creates strong joint dependencies between the features and knockoffs, which allow machine learning algorithms to partially or fully reconstruct the effect of the features on the response using the knockoffs. To improve the power of knockoffs, we propose generating knockoffs which minimize the reconstructability (MRC) of the features, and we demonstrate our proposal for Gaussian features by showing it is computationally efficient, robust, and powerful. We also prove that certain MRC knockoffs minimize a natural definition of estimation error in Gaussian linear models. Furthermore, in an extensive set of simulations, we find many settings with correlated features in which MRC knockoffs dramatically outperform MAC-minimizing knockoffs and no settings in which MAC-minimizing knockoffs outperform MRC knockoffs by more than a very slight margin. We implement our methods and a host of others from the knockoffs literature in a new open source python package knockpy. 1

show abstract

Section: Simulations On Equicorrelated Designsmentioning

confidence: 67%

“…In this section, we discuss two examples in the literature where SDP knockoffs fail to have any power. First, Xing et al (2019) ran simulations in the setting where X is Gaussian and equicorrelated, and Y | X ∼ N (Xβ, 1). They let n = 1000, p = 300, and vary ρ between 0 and 0.8.…”

Section: F3 Examples From the Literaturementioning

confidence: 99%

Powerful Knockoffs via Minimizing Reconstructability

Asher¹,

Janson²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…A detailed proof of the theorem can be found in [31]. It shows that the symmetric property of the mirror statistics M j for null features is satisfied if we choose c j that minimizes the magnitude of the partial correlation I L j (c), which is equivalent to solving I L j (c) = 0 in linear models.…”

Section: Gaussian Mirror Design For Linear Modelsmentioning

confidence: 94%

“…As shown in Theorem 4 of [31], under weak dependence assumption of M j s, we have E[F DP (τ q )] < q as p goes to infinity, i.e., the FDR is asymptotically controlled under the predefined threshold q.…”

Section: Gaussian Mirror Design For Linear Modelsmentioning

confidence: 95%

Neural Gaussian Mirror for Controlled Feature Selection in Neural Networks

Xing,

Gui,

Dai

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) have become increasingly popular and achieved outstanding performance in predictive tasks. However, the DNN framework itself cannot inform the user which features are more or less relevant for making the prediction, which limits its applicability in many scientific fields. We introduce neural Gaussian mirrors (NGMs), in which mirrored features are created, via a structured perturbation based on a kernel-based conditional dependence measure, to help evaluate feature importance. We design two modifications of the DNN architecture for incorporating mirrored features and providing mirror statistics to measure feature importance. As shown in simulated and real data examples, the proposed method controls the feature selection error rate at a predefined level and maintains a high selection power even with the presence of highly correlated features.

show abstract

“…Knockoff filter is introduced in (Barber et al, 2015) with exact control of FDR and can be extended in a model-free way in (Candès et al, 2016). Recently, methods based on mirror statistics are put forward under this topic: (Xing et al, 2019) creates Gaussian mirror variables for all features that get rid of the conditional correlation within each mirrored pair; (Dai et al, 2020) utilizes the data splitting and multiple splitting techniques to ensure the recovery of feature importance with stability.…”

Section: S|mentioning

confidence: 99%

ADAGES: adaptive aggregation with stability for distributed feature selection

Gui

2020

Preprint

View full text Add to dashboard Cite

In this era of "big" data, not only the large amount of data keeps motivating distributed computing, but concerns on data privacy also put forward the emphasis on distributed learning. To conduct feature selection and to control the false discovery rate in a distributed pattern with multi-machines or multi-institutions, an efficient aggregation method is necessary. In this paper, we propose an adaptive aggregation method called ADAGES which can be flexibly applied to any machine-wise feature selection method. We will show that our method is capable of controlling the overall FDR with a theoretical foundation while maintaining power as good as the Union aggregation rule in practice.

show abstract

Controlling False Discovery Rate Using Gaussian Mirrors

Cited by 21 publications

References 36 publications

Powerful Knockoffs via Minimizing Reconstructability

Powerful Knockoffs via Minimizing Reconstructability

Neural Gaussian Mirror for Controlled Feature Selection in Neural Networks

ADAGES: adaptive aggregation with stability for distributed feature selection

Contact Info

Product

Resources

About