Rates of Convergence of the Functional $k$-Nearest Neighbor Estimate

Biau, Gérard; Cérou, Frédéric; Guyader, Arnaud

doi:10.1109/tit.2010.2040857

Cited by 77 publications

(72 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The convergence and consistency properties of trees and random forests have been studied by, among others, Biau [2012], Biau et al [2008], Breiman [2004], Breiman et al [1984], Meinshausen [2006], Scornet et al [2015], Wager and Walther [2015], and Zhu et al [2015]. Meanwhile, their sampling variability has been analyzed by Duan [2011], Lin and Jeon [2006], Mentch and Hooker [2016], Sexton andLaake [2009], and.…”

Section: Theoretical Backgroundmentioning

confidence: 99%

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Wager

Athey

2018

Journal of the American Statistical Association

1,793

1,780

View full text Add to dashboard Cite

Many scientific and engineering challenges-ranging from personalized medicine to customized marketing recommendations-require an understanding of treatment effect heterogeneity. In this paper, we develop a non-parametric causal forest for estimating heterogeneous treatment effects that extends Breiman's widely used random forest algorithm. In the potential outcomes framework with unconfoundedness, we show that causal forests are pointwise consistent for the true treatment effect, and have an asymptotically Gaussian and centered sampling distribution. We also discuss a practical method for constructing asymptotic confidence intervals for the true treatment effect that are centered at the causal forest estimates. Our theoretical results rely on a generic Gaussian theory for a large family of random forest algorithms. To our knowledge, this is the first set of results that allows any type of random forest, including classification and regression forests, to be used for provably valid statistical inference. In experiments, we find causal forests to be substantially more powerful than classical methods based on nearest-neighbor matching, especially in the presence of irrelevant covariates.

show abstract

Section: Theoretical Backgroundmentioning

confidence: 99%

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Wager

Athey

2018

Journal of the American Statistical Association

1,793

1,780

View full text Add to dashboard Cite

show abstract

“…For b-NN , the recent papers by Biau and colleagues (15, 17, 19, 20) showed consistency under general conditions when operating as regression estimators. Therefore, these machines are also consistent when acting as probability machines.…”

Section: Methodsmentioning

confidence: 96%

Probability Machines

Malley¹,

Kruppa²,

Dasgupta³

et al. 2012

Methods Inf Med

190

108

View full text Add to dashboard Cite

Summary Background Most machine learning approaches only provide a classification for binary responses. However, probabilities are required for risk estimation using individual patient characteristics. It has been shown recently that every statistical learning machine known to be consistent for a nonparametric regression problem is a probability machine that is provably consistent for this estimation problem. Objectives The aim of this paper is to show how random forests and nearest neighbors can be used for consistent estimation of individual probabilities. Methods Two random forest algorithms and two nearest neighbor algorithms are described in detail for estimation of individual probabilities. We discuss the consistency of random forests, nearest neighbors and other learning machines in detail. We conduct a simulation study to illustrate the validity of the methods. We exemplify the algorithms by analyzing two well-known data sets on the diagnosis of appendicitis and the diagnosis of diabetes in Pima Indians. Results Simulations demonstrate the validity of the method. With the real data application, we show the accuracy and practicality of this approach. We provide sample code from R packages in which the probability estimation is already available. This means that all calculations can be performed using existing software. Conclusions Random forest algorithms as well as nearest neighbor approaches are valid machine learning methods for estimating individual probabilities for binary responses. Freely available implementations are available in R and may be used for applications.

show abstract

“…While being intuitive and simple to implement, k-nearest neighbor regression is well-understood from the point of view of theory as well, see e.g. [2], [3], [13], and the references therein for an overview of the most important theoretical results. These theoretical results are also justified by empirical studies: for example, in their recent paper, Stensbo-Smidt et al found that nearest neighbor regression outperforms model-based prediction of star formation rates [30], while Hu et al showed that a model based on k-nearest neighbor regression is able to estimate the capacity of lithium-ion batteries [19].…”

Section: Ecknn: K-nearest Neighbor Regression With Error Correctionmentioning

confidence: 99%

Drug–target interaction prediction with Bipartite Local Models and hubness-aware regression

Búza

Peška

2017

Neurocomputing

View full text Add to dashboard Cite

Rates of Convergence of the Functional $k$-Nearest Neighbor Estimate

Cited by 77 publications

References 17 publications

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Probability Machines

Drug–target interaction prediction with Bipartite Local Models and hubness-aware regression

Contact Info

Product

Resources

About