The Balance‐Sample Size Frontier in Matching Methods for Causal Inference

Abstract

We propose a simplified approach to matching for causal inference that simultaneously optimizes balance (similarity between the treated and control groups) and matched sample size. Existing approaches either fix the matched sample size and maximize balance or fix balance and maximize sample size, leaving analysts to settle for suboptimal solutions or attempt manual optimization by iteratively tweaking their matching method and rechecking balance. To jointly maximize balance and sample size, we introduce the matching frontier, the set of matching solutions with maximum possible balance for each sample size. Rather than iterating, researchers can choose matching solutions from the frontier for analysis in one step. We derive fast algorithms that calculate the matching frontier for several commonly used balance metrics. We demonstrate this approach with analyses of the effect of sex on judging and job training programs that show how the methods we introduce can extract new knowledge from existing data sets. comments on an early version were especially helpful. Software to implement the methods proposed in this article can be found at http://projects.iq.harvard.edu/frontier.

Leveraging Peer‐to‐Peer Connections to Increase Voter Participation in Local Elections

“…To obtain better effectiveness estimates with less bias, I use a technique that relies on a matching frontier metric to optimize balance between the groups of eligible voters who received an email and those who did not, as well as the sample size from which inferences are to be made (King, Lucas, and Nielson ). Every treated voter—those who received a VoterCircle email—is matched with the closest nontreated voter—those who did not receive a VoterCircle email—by minimizing the Mahalanobis distance between a treated and nontreated voter on all explanatory variables.…”

“…With the recognition of these limitations, I attempt to approximate a randomized field experiment by matching the treatment sample to the most reasonable set of counterfactual individuals who did not receive a VoterCircle email. I use the latest strategy in matching developed and described in King, Lucas, and Nielson (). While the VoterCircle deployment lacks randomization, this estimation strategy provides the soundest basis for making inferences as to the effectiveness of VoterCircle in a way that best approximates experimental standards given the limitations of the reality of previous deployment.…”

The rippling dynamics of valenced messages in naturalistic youth chat

“…Since we could not eliminate all possible confounds in one specification, we complement the analysis of the main text with a number of alternative specifications and specialized tests (all reported in the SI). We took advantage of several recent advances that made it possible to quantify the extent of assumption violations and to eliminate causally invalid analyses (Arceneaux, Gerber, & Green, 2006;Blackwell et al, 2009;Ho et al, 2017;Iacus et al, 2012Iacus et al, , 2015King, Lucas, & Nielsen, 2014;Schutte & Donnay, 2014). Specifically, we tested the ability of our method to establish (1) the robustness of socially mediated self-influence to selection bias due to the confounding selection of low-quality matches, (2) its robustness to selection bias due to nonrandom deletion of data, (3) its robustness to details of the model specification, and (4) its robustness to faulty instruments.…”

No! Formal Theory, Causal Inference, and Big Data Are Not Contradictory Trends in Political Science

“…The central principle of methods such as matching is to compare treated and untreated observations that were as similar as possible before treatment. Achieving balance, however, requires pruning the data of observations, creating a tradeoff between bias and variance (King, Lucas, and Nielsen 2014 ). Higher standards for balance will reduce bias but may leave data too sparse for useful inference.…”