Lasso adjustments of treatment effect estimates in randomized experiments

Abstract

We provide a principled way for investigators to analyze randomized experiments when the number of covariates is large. Investigators often use linear multivariate regression to analyze randomized experiments instead of simply reporting the difference of means between treatment and control groups. Their aim is to reduce the variance of the estimated treatment effect by adjusting for covariates. If there are a large number of covariates relative to the number of observations, regression may perform poorly because of overfitting. In such cases, the least absolute shrinkage and selection operator (Lasso) may be helpful. We study the resulting Lasso-based treatment effect estimator under the Neyman-Rubin model of randomized experiments. We present theoretical conditions that guarantee that the estimator is more efficient than the simple difference-of-means estimator, and we provide a conservative estimator of the asymptotic variance, which can yield tighter confidence intervals than the difference-ofmeans estimator. Simulation and data examples show that Lassobased adjustment can be advantageous even when the number of covariates is less than the number of observations. Specifically, a variant using Lasso for selection and ordinary least squares (OLS) for estimation performs particularly well, and it chooses a smoothing parameter based on combined performance of Lasso and OLS.randomized experiment | Neyman-Rubin model | average treatment effect | high-dimensional statistics | Lasso R andomized experiments are widely used to measure the efficacy of treatments. Randomization ensures that treatment assignment is not influenced by any potential confounding factors, both observed and unobserved. Experiments are particularly useful when there is no rigorous theory of a system's dynamics, and full identification of confounders would be impossible. This advantage was cast elegantly in mathematical terms in the early 20th century by Jerzy Neyman, who introduced a simple model for randomized experiments, which showed that the difference of average outcomes in the treatment and control groups is statistically unbiased for the average treatment effect (ATE) over the experimental sample (1).However, no experiment occurs in a vacuum of scientific knowledge. Often, baseline covariate information is collected about individuals in an experiment. Even when treatment assignment is not related to these covariates, analyses of experimental outcomes often take them into account with the goal of improving the accuracy of treatment effect estimates. In modern randomized experiments, the number of covariates can be very large-sometimes even larger than the number of individuals in the study. In clinical trials overseen by regulatory bodies like the Food and Drug Administration and the Medicines and Healthcare products Regulatory Agency, demographic and genetic information may be recorded about each patient. In applications in the tech industry, where randomization is often called A/B testing, there is often a huge amount of behavioral data ...

On Mitigating the Analytical Limitations of Finely Stratified Experiments

“…Practical limitations hinder the actualization of theoretical benefits: an issue which we now seek to mitigate. Recent work by Aronow and Middleton (2013), Lin (2013), Bloniarz et al (2016), Lu (2016) and Fogarty (2018) among others has shown how regression adjustment can be utilized to provide improved estimators for the average treatment effect in various experimental designs. In this work, we illustrate how regression adjustment can be utilized to yield improved variance estimators in finely stratified experiments while using the classical difference-in-means estimator for the average treatment effect, hence reducing estimator variance through fine stratification while preserving the so-called 'hands above the table' analysis (Freedman, 2008;Lin, 2013); see also Cox (2007) and Rosenbaum (2010), section 6, for more on the importance of transparency for facilitating critical discussion.…”

Moving beyond the conventional stratified analysis to estimate an overall treatment efficacy with the data from a comparative randomized clinical study

“…In practice, it is desirable to automatically select the stratification factors based on the observed data to avoid subjectivity in the analysis. Recently, Tian et al and Bloniarz et al generalized the augmentation method originally proposed by Zhang et al to efficiently select relevant baseline covariates among a set of prespecified candidates under an unconditional setting for adjusting the consistent two‐sample estimator. It is not clear how to apply this idea to handle the case when there is a potential imbalance with respect to a large set of stratification factors to avoid overstratification.…”

Drawing causal inference from Big Data

“…Bloniarz et al (12) present "Lasso adjustments of treatment effect estimates in randomized experiments," providing a "Lasso" method for overcoming the limitations of linear multivariate regression for dealing with large numbers of covariates.…”