Valid post-selection inference

Berk, Richard A.; Brown, Lawrence D.; Buja, Andreas; Zhang, Kai; Zhao, Linda

doi:10.1214/12-aos1077

Cited by 500 publications

(735 citation statements)

References 34 publications

Supporting

Mentioning

721

Contrasting

Order By: Relevance

“…In general, the challenge in using adaptive methods as the basis for valid statistical inference is that selection bias can be difficult to quantify; see Berk et al [2013], Chernozhukov et al [2015], Taylor and Tibshirani [2015], and references therein for recent advances. In this paper, pairing "honest" trees with the subsampling mechanism of random forests enabled us to accomplish this goal in a simple yet principled way.…”

Section: Discussionmentioning

confidence: 99%

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Wager

Athey

2018

Journal of the American Statistical Association

1,793

1,780

View full text Add to dashboard Cite

Many scientific and engineering challenges-ranging from personalized medicine to customized marketing recommendations-require an understanding of treatment effect heterogeneity. In this paper, we develop a non-parametric causal forest for estimating heterogeneous treatment effects that extends Breiman's widely used random forest algorithm. In the potential outcomes framework with unconfoundedness, we show that causal forests are pointwise consistent for the true treatment effect, and have an asymptotically Gaussian and centered sampling distribution. We also discuss a practical method for constructing asymptotic confidence intervals for the true treatment effect that are centered at the causal forest estimates. Our theoretical results rely on a generic Gaussian theory for a large family of random forest algorithms. To our knowledge, this is the first set of results that allows any type of random forest, including classification and regression forests, to be used for provably valid statistical inference. In experiments, we find causal forests to be substantially more powerful than classical methods based on nearest-neighbor matching, especially in the presence of irrelevant covariates.

show abstract

Section: Discussionmentioning

confidence: 99%

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Wager

Athey

2018

Journal of the American Statistical Association

1,793

1,780

View full text Add to dashboard Cite

show abstract

“…31 and 32. In Berk et al (32), an intuitive and practical strategy has been proposed, but it fails when the dimension of the variables is high. To our knowledge, there is no clear solution in statistics for post-model-selection inference problem for high-dimensional data; it is an interesting problem that may warrant future investigation.…”

Section: Methodsmentioning

confidence: 99%

Pattern discovery and cancer gene identification in integrated cancer genomic data

Wang

Seshan

et al. 2013

Proc. Natl. Acad. Sci. U.S.A.

381

371

View full text Add to dashboard Cite

Large-scale integrated cancer genome characterization efforts including the cancer genome atlas and the cancer cell line encyclopedia have created unprecedented opportunities to study cancer biology in the context of knowing the entire catalog of genetic alterations. A clinically important challenge is to discover cancer subtypes and their molecular drivers in a comprehensive genetic context. Curtis et al. [Nature (2012) 486(7403):346-352] has recently shown that integrative clustering of copy number and gene expression in 2,000 breast tumors reveals novel subgroups beyond the classic expression subtypes that show distinct clinical outcomes. To extend the scope of integrative analysis for the inclusion of somatic mutation data by massively parallel sequencing, we propose a framework for joint modeling of discrete and continuous variables that arise from integrated genomic, epigenomic, and transcriptomic profiling. The core idea is motivated by the hypothesis that diverse molecular phenotypes can be predicted by a set of orthogonal latent variables that represent distinct molecular drivers, and thus can reveal tumor subgroups of biological and clinical importance. Using the cancer cell line encyclopedia dataset, we demonstrate our method can accurately group cell lines by their cell-of-origin for several cancer types, and precisely pinpoint their known and potential cancer driver genes. Our integrative analysis also demonstrates the power for revealing subgroups that are not lineage-dependent, but consist of different cancer types driven by a common genetic alteration. Application of the cancer genome atlas colorectal cancer data reveals distinct integrated tumor subtypes, suggesting different genetic pathways in colon cancer progression.A major goal of many cancer genome projects is to characterize key genetic alterations in cancer and discover therapeutic targets through comprehensive genomic profiling of the cancer genome. The Cancer Genome Atlas (TCGA) studies have unveiled the genetic landscape of several cancer types by whole-genome and whole-exome sequencing, DNA copy number profiling, promoter methylation profiling, and mRNA expression profiling in a large number of tumors (1-5). Complementary to the tumor project, the Cancer Cell Line Encyclopedia (CCLE) (6) and the Sanger cell line project (7) has cataloged a compilation of genetic and molecular data in almost 1,000 human cancer cell lines, coupled with pharmacological profiles for a large panel of anticancer drugs. These large-scale integrative genomic efforts have been geared toward comprehensively cataloging individual genomic alterations, analogous to a reverse-engineering process where thousands of individual cancer genomes are taken apart to shed light on common biological principles. Unfortunately, cancer genomes exhibit considerable heterogeneity with abnormalities occurring in different genes among different individuals, posing a great challenge to identify those genes with functional importance and therapeutic implications. Thus, there is a...

show abstract

“…This fundamental shift in focus allows us to circumvent, without contradicting, the impossibility results of Leeb and Pötscher. Valid post-selection inference has attracted considerable attention during the preparation of this paper: in contexts and with methods quite different from ours, contributions have been made by Belloni, Chernozhukov, and Wei (2013), Berk, Brown, Buja, Zhang, and Zhao (2013), Zhang and Zhang (2014), Efron (2014), van de Geer, Buhlmann, Ritov, and Dezeure (2014), and Belloni, Chernozhukov, Fernandez-Val, and Hansen (2014), among others.…”

Section: Introductionmentioning

confidence: 99%

Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations

2013

View full text Add to dashboard Cite

This paper concerns robust inference on average treatment effects following model selection. Under selection on observables, we construct confidence intervals using a doubly-robust estimator that are robust to model selection errors and prove their uniform validity over a large class of models that allows for multivalued treatments with heterogeneous effects and selection amongst (possibly) more covariates than observations. The semiparametric efficiency bound is attained under appropriate conditions. Precise conditions are given for any model selector to yield these results, and we specifically propose the group lasso, which is apt for treatment effects, and derive new results for high-dimensional, sparse multinomial logistic regression. Both a simulation study and revisiting the National Supported Work demonstration show our estimator performs well in finite samples.

show abstract

Valid post-selection inference

Cited by 500 publications

References 34 publications

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Pattern discovery and cancer gene identification in integrated cancer genomic data

Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations

Contact Info

Product

Resources

About