Double/debiased machine learning for treatment and structural parameters

Abstract

SummaryWe revisit the classic semi-parametric problem of inference on a low-dimensional parameter θ 0 in the presence of high-dimensional nuisance parameters η 0 . We depart from the classical setting by allowing for η 0 to be so high-dimensional that the traditional assumptions (e.g. Donsker properties) that limit complexity of the parameter space for this object break down. To estimate η 0 , we consider the use of statistical or machine learning (ML) methods, which are particularly well suited to estimation in modern, very high-dimensional cases. ML methods perform well by employing regularization to reduce variance and trading off regularization bias with overfitting in practice. However, both regularization bias and overfitting in estimating η 0 cause a heavy bias in estimators of θ 0 that are obtained by naively plugging ML estimators of η 0 into estimating equations for θ 0 . This bias results in the naive estimator failing to be N −1/2 consistent, where N is the sample size. We show that the impact of regularization bias and overfitting on estimation of the parameter of interest θ 0 can be removed by using two simple, yet critical, ingredients: (1) using Neyman-orthogonal moments/scores that have reduced sensitivity with respect to nuisance parameters to estimate θ 0 ; (2) making use of cross-fitting, which provides an efficient form of data-splitting. We call the resulting set of methods double or debiased ML (DML). We verify that DML delivers point estimators that concentrate in an N −1/2 -neighbourhood of the true parameter values and are approximately unbiased and normally distributed, which allows construction of valid confidence statements. The generic statistical theory of DML is elementary and simultaneously relies on only weak theoretical requirements, which will admit the use of a broad array of modern ML methods for estimating the nuisance parameters, such as random forests, lasso, ridge, deep neural nets, boosted trees, and various hybrids and ensembles of these methods. We illustrate the general theory by applying it to provide theoretical properties of the following: DML applied to learn the main regression parameter in a partially linear regression model; DML applied to learn the coefficient on an endogenous variable in a partially linear instrumental variables model; DML applied to learn the average treatment effect and the average treatment effect on the treated under unconfoundedness; DML applied

Approximate Residual Balancing: Debiased Inference of Average Treatment Effects in High Dimensions

“…(), Chernozhukov et al . (), Farrell () and Van der Laan and Rose () built on the work of Robins et al . () and discussed how a doubly robust approach to average treatment effect estimation in high dimensions can also be used to compensate for the bias of regularized regression adjustments.…”

Machine Learning: An Applied Econometric Approach

“…For example, Lee, Lessler, and Stuart (2010) use machine-learning algorithms to estimate the propensity score. Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016) take care of high-dimensional controls in treatment effect estimation by solving two simultaneous prediction problems, one in the outcome and one in the treatment equation.…”

Estimation of average treatment effects among multiple treatment groups by using an ensemble approach

“…The enDR developed in this article coupled with the optimal GPS method (ie, Opt MinMax ) provided much improved estimates for ATEs. A reviewer has brought a double/debiased machine learning (ML) method to our attention . Double ML method uses the Neyman orthogonal estimating equation and sample splitting strategy to estimate ATE for two groups .…”

“…A reviewer has brought a double/debiased machine learning (ML) method to our attention . Double ML method uses the Neyman orthogonal estimating equation and sample splitting strategy to estimate ATE for two groups . ML methods are used to estimate the nuisance relationship between X and T , as well as X and Y using one part of the data.…”