Rerandomization to improve covariate balance in experiments

Morgan, Kari Lock; Rubin, Donald B.

doi:10.1214/12-aos1008

Cited by 291 publications

(506 citation statements)

References 61 publications

Supporting

Mentioning

487

Contrasting

Order By: Relevance

“…Yet this is of little help in finite samples. As Fisher (1926) noted: "Most experimenters on carrying out a random assignment will be shocked to find how far from equally the plots distribute themselves," quoted in Morgan and Rubin (2012 the means over all possible randomizations) of the observed causes of Y in the control and treatment groups at the point of assignment, the null hypothesis is (presumably, as judged by the typical balance test) that the two vectors are identical, with the alternative being that they are not. But if the randomization has been correctly done the null hypothesis is true by construction (see e.g.…”

Section: Sample Size Balance and Precisionmentioning

confidence: 99%

Understanding and Misunderstanding Randomized Controlled Trials

Deaton¹,

Cartwright²

2016

167

151

View full text Add to dashboard Cite

RCTs would be more useful if there were more realistic expectations of them and if their pitfalls were better recognized. For example, and contrary to many claims in the applied literature, randomization does not equalize everything but the treatment across treatments and controls, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) confounders. Estimates apply to the trial sample only, sometimes a convenience sample, and usually selected; justification is required to extend them to other groups, including any population to which the trial sample belongs. Demanding "external validity" is unhelpful because it expects too much of an RCT while undervaluing its contribution. Statistical inference on ATEs involves hazards that are not always recognized. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon and not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not "what works," but "why things work".

show abstract

Section: Sample Size Balance and Precisionmentioning

confidence: 99%

Understanding and Misunderstanding Randomized Controlled Trials

Deaton¹,

Cartwright²

2016

167

151

View full text Add to dashboard Cite

show abstract

“…1-9, © 2015 INFORMS of our work. For such cases, apart from randomization, two prominent methods are pairwise matching for controlled trials (see Rubin 1985 andGreevy et al 2004) and re-randomization as proposed in Morgan and Rubin 2012. 1 The finite selection model (FSM) proposed by Morris (1979) can also be used for this purpose. In comparisons explored in §4, we find that the balance produced by our proposed optimization-based approach greatly improves on both randomization and these methods.…”

Section: Introductionmentioning

confidence: 99%

The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples

Bertsimas

Johnson

Kallus

2015

Operations Research

View full text Add to dashboard Cite

Random assignment, typically seen as the standard in controlled trials, aims to make experimental groups statistically equivalent before treatment. However, with a small sample, which is a practical reality in many disciplines, randomized groups are often too dissimilar to be useful. We propose an approach based on discrete linear optimization to create groups whose discrepancy in their means and variances is several orders of magnitude smaller than with randomization. We provide theoretical and computational evidence that groups created by optimization have exponentially lower discrepancy than those created by randomization and that this allows for more powerful statistical inference.

show abstract

“…As an alternative to blocking, some advocate rerandomization when a given randomization results in poor balance in observed covariates (26,27). Rerandomization restricts the randomization scheme, as assignments with poor balance are ruled out.…”

Section: Discussionmentioning

confidence: 99%

Improving massive experiments with threshold blocking

Higgins

Sävje

Sekhon

2016

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Inferences from randomized experiments can be improved by blocking: assigning treatment in fixed proportions within groups of similar units. However, the use of the method is limited by the difficulty in deriving these groups. Current blocking methods are restricted to special cases or run in exponential time; are not sensitive to clustering of data points; and are often heuristic, providing an unsatisfactory solution in many common instances. We present an algorithm that implements a widely applicable class of blocking-threshold blockingthat solves these problems. Given a minimum required group size and a distance metric, we study the blocking problem of minimizing the maximum distance between any two units within the same group. We prove this is a nondeterministic polynomial-time hard problem and derive an approximation algorithm that yields a blocking where the maximum distance is guaranteed to be, at most, four times the optimal value. This algorithm runs in O(n log n) time with O(n) space complexity. This makes it, to our knowledge, the first blocking method with an ensured level of performance that works in massive experiments. Whereas many commonly used algorithms form pairs of units, our algorithm constructs the groups flexibly for any chosen minimum size. This facilitates complex experiments with several treatment arms and clustered data. A simulation study demonstrates the efficiency and efficacy of the algorithm; tens of millions of units can be blocked using a desktop computer in a few minutes.experimental design | blocking | big data | causal inference

show abstract

Rerandomization to improve covariate balance in experiments

Cited by 291 publications

References 61 publications

Understanding and Misunderstanding Randomized Controlled Trials

Understanding and Misunderstanding Randomized Controlled Trials

The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples

Improving massive experiments with threshold blocking

Contact Info

Product

Resources

About