Inference for Two-stage Experiments under Covariate-Adaptive Randomization

Abstract

This paper studies inference in two-stage randomized experiments with covariate-adaptive randomization. Here, by a two-stage randomized experiment, we mean one in which clusters (e.g., households, schools, or graph partitions) are first randomly assigned to different levels of treated fraction and then units within each treated clusters are randomly assigned to treatment or control according to its selected treated fraction; by covariate-adaptive randomization, we mean randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve "balance" within each stratum. We study estimation and inference of this design under two different asymptotic regimes: "small strata" and "large strata", which enable us to study a wide range of commonly used designs from the empirical literature. We establish conditions under which our estimators are consistent and asymptotically normal and construct consistent estimators of their corresponding asymptotic variances. Combining these results establishes the asymptotic validity of tests based on these estimators. We argue that ignoring covariate information at the design stage can lead to efficiency loss, and commonly used inference methods that ignore or improperly use covariate information can lead to either conservative or invalid inference. Then, we apply our results to studying optimal use of covariate information in twostage designs, and show that a certain generalized matched-pair design achieves minimum asymptotic variance for each proposed estimator. A simulation study and empirical application confirm the practical relevance of our theoretical results.