Matching for balance, pairing for heterogeneity in an observational study of the effectiveness of for-profit and not-for-profit high schools in Chile

Abstract

are for-profit enterprises not different in concept than a restaurant or retail store. Whether schools should be allowed to profit is an intensely controversial issue in Chile. On the one hand, supporters of for-profit schools argue that they have incentives for efficiency and innovation, and that this in turn results in better education. Opposing this view, detractors say that, in reducing costs, for-profit schools tend to also reduce the quality of education and that one cannot allow a desire for profits to take precedence over the quality of a child's education [see Elacqua (2009) for further discussion]. In 2011, in support of the latter view, and in part with the goal of ending forprofit education in Chile, thousands of students rallied through the streets demanding a change in the model of education and better opportunities.Here, we compare the 2006 academic test performance of Chilean students who entered for-profit private high schools and students who entered not-for-profit private high schools. All of these students were in government run primary/middle schools in Santiago in 2004 and subsequently moved to private high schools. We have test scores at baseline in 2004 in language (Spanish), mathematics, natural science and social science, and we have outcome test scores in 2006 in language and mathematics. In addition, we have extensive data about parents and children in 2004, such as the education of the parents, their income, the number of books at home and so on, recorded in an observed covariate x. An obvious concern is that even after adjusting for a high-dimensional observed covariate x, children in different types of schools may differ in terms of some other covariate u that was not observed, and differences in u may bias the comparison.The test scores come from the SIMCE, the Spanish acronym for "System of Measurement of Quality in Education." For the same students, we use test scores for the 8th grade of primary school in 2004 and the second year of high school in 2006. For the typical student, these are test scores at ages 14 and 16. For-profit and not-for-profit are determined by the official definitions of the Chilean IRS based on the institutional identification number (RUT).Do profits boost or depress test scores in similar students? Or are profits irrelevant to test scores? 1.2. Matching for covariate balance, pairing for heterogeneity. To be credible, the comparison must compare children in not-for-profit schools (the treated group) to children similar at baseline in for-profit schools (the control group), and there are many ways the children may differ. It is typically difficult to match closely for all coordinates of a high-dimensional observed covariate x, but it is often not difficult to create matched treated and control groups with similar distributions of x. For instance, if x consisted of 20 binary covariates, it would distinguish 2 20 or about a million categories of students, so it would be very difficult to match thousands of students exactly for all 20 covariates. However, it...

Large, Sparse Optimal Matching With Refined Covariate Balance in an Observational Study of the Health Outcomes Produced by New Surgeons

“…Refined covariate balance in such a network would obtain pairs that are close on the key covariates while balancing many nominal categories. As discussed by Zubizarreta et al (2014), a match that reduces the heterogeneity of matched pair differences in outcomes, perhaps by matching closely for predictors of those outcomes, will both increase the power of a randomization test of no effect and increase its insensitivity to unmeasured biases.…”

“…For discussion of matching, see Baiocchi et al (2012), Hansen et al (2006, 2007), Heller et al (2009), Lu et al (2011), Rosenbaum (1989, 2010), Rosenbaum and Rubin (1985), Stuart (2010), Yang et al (2012), and Zubizarreta et al (2011, 2014). For recent applications of optimal matching, see Silber et al (2013) and Neuman et al (2014).…”

Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System

“…We describe the matching algorithm in greater detail in Section 3. The match is based on integer programming which allows us to enforce different forms of balance for different covariates (Zubizarreta 2012;Zubizarreta et al 2014). This is relevant since we tailored the constraints for each covariate.…”

“…Common matching methods attempt to achieve covariate balance indirectly, by finding treated and control units that are close on a summary measure of the covariates such as the Mahalanobis distance or the propensity score (seeStuart 2010 andLu et al 2011 for reviews). Unlike these matching methods, cardinality matching uses the original covariates to match units and directly balance their covariate distributions(Zubizarreta et al 2014). Specifically, by solving an integer programming problem, cardinality matching finds the largest matched sample that satisfies the investigator's specifications for covariate balance.…”

Full matching approach to instrumental variables estimation with application to the effect of malaria on stunting

“…For example, sulpadoxine pyrimethamine vs. placebo was randomly assigned as part of a randomized trial. However, we still have chosen to match on all the covariates because each covariate may be associated with the outcome and matching a covariate that is associated with the outcome increases efficiency and reduces sensitivity to unobserved biases (Rosenbaum, 2005;Zubizarreta, Paredes and Rosenbaum, 2014). Furthermore, Rubin (2009) argues for erring on the side of being inclusive when deciding which variables to match on (i.e.…”