Multiple Testing for Composite Null with FDR Control Guarantee

Abstract

False discovery rate (FDR) controlling procedures provide important statistical guarantees for reproducibility in signal identification experiments with multiple hypotheses testing. In many recent applications, the same set of candidate features are studied in multiple independent experiments. For example, experiments repeated at different facilities and with different cohorts, and association studies with the same candidate features but different outcomes of interest. These studies provide us opportunities to identify signals by considering the experiments jointly. We study the question of how to provide reproducibility guarantees when we test composite null hypotheses on multiple features. Specifically, we test the unions of the null hypotheses from multiple experiments. We present a knockoff-based variable selection method to identify mutual signals from multiple independent experiments, with a finite sample size FDR control guarantee. We demonstrate the performance of this method with numerical studies and applications in analyzing crime data and TCGA data.