The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation

Abstract

A basic feature of many field experiments is that investigators are only able to randomize clusters of individuals-such as households, communities, firms, medical practices, schools or classroomseven when the individual is the unit of interest. To recoup the resulting efficiency loss, some studies pair similar clusters and randomize treatment within pairs. However, many other studies avoid pairing, in part because of claims in the literature, echoed by clinical trials standards organizations, that this matched-pair, cluster-randomization design has serious problems. We argue that all such claims are unfounded. We also prove that the estimator recommended for this design in the literature is unbiased only in situations when matching is unnecessary; its standard error is also invalid. To overcome this problem without modeling assumptions, we develop a simple design-based estimator with much improved statistical properties. We also propose a model-based approach that includes some of the benefits of our design-based estimator as well as the estimator in the literature. Our methods also address individual-level noncompliance, which is common in applications but not allowed for in most existing methods. We show that from the perspective of bias, efficiency, power, robustness or research costs, and in large or small samples, pairing should be used in cluster-randomized experiments whenever feasible; failing to do so is equivalent to discarding a considerable fraction of one's data. We develop these techniques in the context of a randomized evaluation we are conducting of the Mexican Universal Health Insurance Program.

Impact of a Wellness Portal on the Delivery of Patient-Centered Preventive Care

“…Become informed, activated patients: 8 Selection criteria included clinician experience with the existing patient registry (PSRS), inclusion of both young children (age Ͻ6) and adults (age Ͼ50) in the patient panel, and no previous experience with the Wellness Portal.…”

Avoiding Randomization Failure in Program Evaluation, with Application to the Medicare Health Support Program

“…The result is that matched pairs is preferable in most situations. 6 The exceptions to this rule are not statistical but administrative. For example, in drug trials for serious but rare diseases, eligible patients may become available only sporadically, and medical personnel cannot wait until a good match is found to treat one patient.…”

“…(Studying the complier causal effect-that is, the effect among those who would accept treatment if they were assigned to the treated group and would not take treatment if assigned to the control group-requires special statistical procedures). 6 In the MHS experiment, because of the way the analysis was conducted, beneficiaries who received an intent-to-treat random assignment did not necessarily have the opportunity to receive treatment. The data analysis procedure included and excluded beneficiaries at different points in the study, including after randomization, based on the eligibility criteria.…”

Impact Evaluation of Development Interventions:

“…Without prior matching both may end up in one of these two groups, unbalancing the sample. When possible to conduct, matched pair randomization has the potential to improve sample power, efficiency, robustness, and increase power (Imai et al 2009). With stratification, participants are divided into groups (strata)-such as low, medium, and high income-and then randomization is conducted for each group.…”