Balancing covariates in randomized experiments with the Gram-Schmidt Walk design

Harshaw, Christopher; Sävje, Fredrik; Spielman, Daniel A.; Zhang, Peng

doi:10.48550/arxiv.1911.03071

Cited by 10 publications

(22 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In practice with finite samples, considered optimizing the threshold to minimize a certain quantile of treatment effect estimation error under some model assumptions on the potential outcomes. Recently, Harshaw et al (2019) proposed a Gram-Schmidt Walk design involving an explicit parameter, which plays a similar role as the rerandomization threshold, for controlling the trade-off between covariate balance and robustness.…”

Section: Covariate Balance Criteria Using the Mahalanobis Distancesmentioning

confidence: 99%

Rejective Sampling, Rerandomization and Regression Adjustment in Survey Experiments

Yang¹,

Qu²,

Li³

2021

Preprint

View full text Add to dashboard Cite

Classical randomized experiments, equipped with randomization-based inference, provide assumption-free inference for treatment effects. They have been the gold standard for drawing causal inference and provide excellent internal validity. However, they have also been criticized for questionable external validity, in the sense that the conclusion may not generalize well to a larger population. The randomized survey experiment is a design tool that can help mitigate this concern, by randomly selecting the experimental units from the target population of interest. However, as pointed out by Morgan and Rubin (2012), chance imbalances often exist in covariate distributions between different treatment groups even under completely randomized experiments. Not surprisingly, such covariate imbalances also occur in randomized survey experiments. Furthermore, the covariate imbalances happen not only between different treatment groups, but also between the sampled experimental units and the overall population of interest. In this paper, we propose a two-stage rerandomization design that can actively avoid undesirable covariate imbalances at both the sampling and treatment assignment stages. We further develop asymptotic theory for rerandomized survey experiments, demonstrating that rerandomization provides better covariate balance, more precise treatment effect estimators, and shorter large-sample confidence intervals. We also propose covariate adjustment to deal with remaining covariate imbalances after rerandomization, showing that it can further improve both the sampling and estimated precision. Our work allows general relationship among covariates at the sampling, treatment assignment and analysis stages, and generalizes both rerandomization in classical randomized experiments (Morgan and Rubin 2012) and rejective sampling in survey sampling (Fuller 2009).

show abstract

Section: Covariate Balance Criteria Using the Mahalanobis Distancesmentioning

confidence: 99%

Rejective Sampling, Rerandomization and Regression Adjustment in Survey Experiments

Yang¹,

Qu²,

Li³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…of a collection X i ∈ R d , 1 ≤ i ≤ n, of vectors. This problem is at the heart of a very important application in statistics, dubbed as randomized controlled trials, which is often considered to be the gold standard for clinical trials [KAK19,HSSZ19]. Consider n individuals participating in a randomized study that seeks inference for an additive treatment effect.…”

Section: Introductionmentioning

confidence: 99%

“…To ensure accurate inference based on the response, it is desirable for the groups to have roughly the same covariates. See the very recent work on the design of such randomized controlled experiments by Harshaw, Sävje, Spielman, and Zhang [HSSZ19] (and the references therein) for a more elaborate discussion on this front. Besides its significance in statistics, NPP appears in many other practical applications.…”

Section: Introductionmentioning

confidence: 99%

Algorithmic Obstructions in the Random Number Partitioning Problem

Gamarnik,

Kızıldağ

2021

Preprint

View full text Add to dashboard Cite

We consider the algorithmic problem of finding a near-optimal solution for the number partitioning problem (NPP). This problem appears in many practical applications, including the design of randomized controlled trials, multiprocessor scheduling, and cryptography; and is also of theoretical significance. The NPP possesses the so-called statistical-tocomputational gap: when its input X has distribution N (0, I n ), the optimal value of the NPP is Θ ( √ n2 −n ) w.h.p.; whereas the best polynomial-time algorithm achieves the objective value of only 2 −Θ(log 2 n) , w.h.p.In this paper, we initiate the study of the nature of this gap. Inspired by insights from statistical physics, we study the landscape of the NPP and establish the presence of the Overlap Gap Property (OGP), an intricate geometric property which is known to be a rigorous evidence of an algorithmic hardness for large classes of algorithms. By leveraging the OGP, we establish that (a) any sufficiently stable algorithm, appropriately defined, fails to find a near-optimal solution with energy below 2 −ω(n log −1/5 n) ; and (b) a very natural Markov Chain Monte Carlo dynamics fails for find near-optimal solutions. Our simulation results suggest that the state of the art algorithm achieving the value 2 −Θ(log 2 n) is indeed stable, but formally verifying this is left as an open problem.OGP regards the overlap structure of m−tuples of solutions achieving a certain objective value. When m is constant we prove the presence of OGP for the objective values of order 2 −Θ(n) , and the absence of it in the regime 2 −o(n) . Interestingly, though, by considering overlaps with growing values of m we prove the presence of the OGP up to the level 2 −ω( √ n log n) . Our proof of the failure of stable algorithms at values 2 −ω(n log −1/5 n) employs methods from Ramsey Theory from the extremal combinatorics, and is of independent interest.

show abstract

“…However, even in completely randomized experiments, there is a lot of chance to produce an unbalanced assignment (Rosenberger et al, 2008;Rubin, 2008;Xu and Kalbfleisch, 2010), and the probability of producing such an unbalanced assignment increases with the number of covariates (Morgan and Rubin, 2012;Krieger et al, 2019). It is widely recognized that balancing covariates in the design stage can improve the experiments (Greevy et al, 2004;Kallus, 2018;Harshaw et al, 2020). Fisher (1926) proposed to use stratification to balance several categorical covariates that are most relevant to the outcomes.…”

Section: Introductionmentioning

confidence: 99%

Pair-switching rerandomization

Zhu¹,

Liu²

2021

Preprint

View full text Add to dashboard Cite

Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve the estimation and inference efficiency. However, the acceptance-rejection sampling method used by rerandomization is computationally inefficient. As a result, it is timeconsuming for classical rerandomization to draw numerous independent assignments, which are necessary for constructing Fisher randomization tests. To address this problem, we propose a pair-switching rerandomization method to draw balanced assignments much efficiently. We show that the difference-in-means estimator is unbiased for the average treatment effect and the Fisher randomization tests are valid under pair-switching rerandomization. In addition, our method is applicable in both non-sequentially and sequentially randomized experiments. We conduct comprehensive simulation studies to compare the finite-sample performances of the proposed method and classical rerandomization. Simulation results indicate that pair-switching rerandomization leads to comparable power of Fisher randomization tests and is 4-18 times faster than classical rerandomization. Finally, we apply the pair-switching rerandomization method to analyze two clinical trial data sets, both demonstrating the advantages of our method.

show abstract

Balancing covariates in randomized experiments with the Gram-Schmidt Walk design

Cited by 10 publications

References 43 publications

Rejective Sampling, Rerandomization and Regression Adjustment in Survey Experiments

Rejective Sampling, Rerandomization and Regression Adjustment in Survey Experiments

Algorithmic Obstructions in the Random Number Partitioning Problem

Pair-switching rerandomization

Contact Info

Product

Resources

About