The LOOP Estimator: Adjusting for Covariates in Randomized Experiments

Wu, Edward Ming-Yang; Gagnon-Bartsch, Johann A.

doi:10.1177/0193841x18808003

Cited by 25 publications

(24 citation statements)

References 48 publications

(105 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the analysis, we focused on inferring the average treatment effect by using regression adjustment. It is interesting to extend the discussion to covariate adjustment in more complicated settings, such as high dimensional covariates (Bloniarz et al ., ; Wager et al ., ; Lei and Ding, ), logistic regression for binary outcomes (Zhang et al ., ; Freedman, 2008d; Moore and van der Laan, ; Moore et al ., ) and adjustment using machine learning methods (Bloniarz et al ., ; Wager et al ., ; Wu and Gagnon‐Bartsch, ). It is also important to consider covariate adjustment for general non‐linear estimands (Zhang et al ., ; Jiang et al ., ; Tian et al ., ) and general designs (Middleton, ), such as blocking (Miratrix et al ., ; Bugni et al ., ), matched pairs (Fogarty, ), and factorial designs (Lu, ).…”

Section: Discussionmentioning

confidence: 99%

Rerandomization and Regression Adjustment

Ding

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Summary Randomization is a basis for the statistical inference of treatment effects without strong assumptions on the outcome‐generating process. Appropriately using covariates further yields more precise estimators in randomized experiments. R. A. Fisher suggested blocking on discrete covariates in the design stage or conducting analysis of covariance in the analysis stage. We can embed blocking in a wider class of experimental design called rerandomization, and extend the classical analysis of covariance to more general regression adjustment. Rerandomization trumps complete randomization in the design stage, and regression adjustment trumps the simple difference‐in‐means estimator in the analysis stage. It is then intuitive to use both rerandomization and regression adjustment. Under the randomization inference framework, we establish a unified theory allowing the designer and analyser to have access to different sets of covariates. We find that asymptotically, for any given estimator with or without regression adjustment, rerandomization never hurts either the sampling precision or the estimated precision, and, for any given design with or without rerandomization, our regression‐adjusted estimator never hurts the estimated precision. Therefore, combining rerandomization and regression adjustment yields better coverage properties and thus improves statistical inference. To quantify these statements theoretically, we discuss optimal regression‐adjusted estimators in terms of the sampling precision and the estimated precision, and then measure the additional gains of the designer and the analyser. We finally suggest the use of rerandomization in the design and regression adjustment in the analysis followed by the Huber–White robust standard error.

show abstract

Section: Discussionmentioning

confidence: 99%

Rerandomization and Regression Adjustment

Ding

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

show abstract

“…In the SUTVA setting, adjustment with OLS works best when the adjustment variables are highly correlated with the potential outcomes; that is, the precision improvement largely depends on the prediction accuracy. This fact suggests that predicted outcomes obtained from an arbitrary machine learning model can be used for adjustment, an idea formalized by Wager et al (2016); Wu and Gagnon-Bartsch (2017). Based on ideas from Aronow and Middleton (2013), these papers propose using the estimator…”

Section: Nonparametric Adjustmentsmentioning

confidence: 99%

“…In the observational studies setting, regression adjustments are used to adjust for inherent differences between the covariate distributions of different treatment groups. We heavily borrow tools from that literature, both in the classical regime of using low-dimensional, linear regression estimators (Freedman 2008a,b;Lin 2013;Berk et al 2013) and more recent advancements that can utilize high-dimensional regression and machine learning techniques (Bloniarz et al 2016;Wager et al 2016;Wu and Gagnon-Bartsch 2017;Chernozhukov et al 2018). This recent literature adopts the agnostic perspective that properties of least squares and machine learning estimators can be utilized without assuming the parametric model itself.…”

Section: Introductionmentioning

confidence: 99%

Regression Adjustments for Estimating the Global Treatment Effect in Experiments with Interference

Chin

2019

Journal of Causal Inference

View full text Add to dashboard Cite

Standard estimators of the global average treatment effect can be biased in the presence of interference. This paper proposes regression adjustment estimators for removing bias due to interference in Bernoulli randomized experiments. We use a fitted model to predict the counterfactual outcomes of global control and global treatment. Our work differs from standard regression adjustments in that the adjustment variables are constructed from functions of the treatment assignment vector, and that we allow the researcher to use a collection of any functions correlated with the response, turning the problem of detecting interference into a feature engineering problem. We characterize the distribution of the proposed estimator in a linear model setting and connect the results to the standard theory of regression adjustments under SUTVA. We then propose an estimator that allows for flexible machine learning estimators to be used for fitting a nonlinear interference functional form. We propose conducting statistical inference via bootstrap and resampling methods, which allow us to sidestep the complicated dependences implied by interference and instead rely on empirical covariance structures. Such variance estimation relies on an exogeneity assumption akin to the standard unconfoundedness assumption invoked in observational studies. In simulation experiments, our methods are better at debiasing estimates than existing inverse propensity weighted estimators based on neighborhood exposure modeling. We use our method to reanalyze an experiment concerning weather insurance adoption conducted on a collection of villages in rural China.

show abstract

“…Aronow and Middleton (2013) use this estimator in a design-based framework and note that if true m ^ i is predictive of the observed outcome Y i , then the resulting estimate will improve over the unadjusted estimator. E. Wu and Gagnon-Bartsch (2018) build on this work and suggest estimating the quantity m i = ( t i + c i ) / 2 .…”

Section: Background and Notationmentioning

confidence: 99%

“…However, in cases where the pair assignments are not predictive of the outcome, it is better to ignore the pairing. Both Aronow and Middleton (2013) and E. Wu and Gagnon-Bartsch (2018) present versions of their methods that allow for block randomizations; however, neither of these methods directly address the pair inclusion trade-off.…”

Section: Introductionmentioning

confidence: 99%

Design-Based Covariate Adjustments in Paired Experiments

Gagnon-Bartsch

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

In paired experiments, participants are grouped into pairs with similar characteristics, and one observation from each pair is randomly assigned to treatment. The resulting treatment and control groups should be well-balanced; however, there may still be small chance imbalances. Building on work for completely randomized experiments, we propose a design-based method to adjust for covariate imbalances in paired experiments. We leave out each pair and impute its potential outcomes using any prediction algorithm such as lasso or random forests. This method addresses a unique trade-off that exists for paired experiments. By addressing this trade-off, the method has the potential to improve precision over existing methods.

show abstract

The LOOP Estimator: Adjusting for Covariates in Randomized Experiments

Cited by 25 publications

References 48 publications

Rerandomization and Regression Adjustment

Rerandomization and Regression Adjustment

Regression Adjustments for Estimating the Global Treatment Effect in Experiments with Interference

Design-Based Covariate Adjustments in Paired Experiments

Contact Info

Product

Resources

About