Are private schools better than public schools? Appraisal for Ireland by methods for observational studies

Abstract

In observational studies the assignment of units to treatments is not under control. Consequently, the estimation and comparison of treatment effects based on the empirical distribution of the responses can be biased since the units exposed to the various treatments could differ in important unknown pretreatment characteristics, which are related to the response. An important example studied in this article is the question of whether private schools offer better quality of education than public schools. In order to address this question we use data collected in the year 2000 by OECD for the Programme for International Student Assessment (PISA). Focusing for illustration on scores in mathematics of 15-years old pupils in Ireland, we find that the raw average score of pupils in private schools is higher than of pupils in public schools. However, application of a newly proposed method for observational studies suggests that the less able pupils tend to enroll in public schools, such that their lower scores is not necessarily an indication of bad quality of the public schools. Indeed, when comparing the average score in the two types of schools after adjusting for the enrollment effects, we find quite surprisingly that public schools perform better on average. This outcome is supported by the methods of instrumental variables and latent variables, commonly used by econometricians for analyzing and evaluating social programs.

Statistical Challenges of Administrative and Transaction Data

“…Applying these concepts to the context of administrative data is an interesting idea. If formal randomization is not achievable, the methods reviewed and proposed by Pfeffermann and Landsman (2011) for causal inference in observational studies can be considered.…”

Comparing Inference Methods for Non‐probability Samples

“…Matching attempts to mimic the concept of restricted randomisation as applied in randomised block designs known from experimental design theory. Pfeffermann and Landsman () review existing methods that account for selection bias in observational studies that are based on strong ignorability assumptions regarding treatment assignment. These are techniques like regression, propensity scoring, matching and doubly robust estimators that assume observations are missing at random.…”

“…In their article they also review methods that attempt to avoid strong ignorability assumptions like latent variable models and instrumental models. In addition, Pfeffermann and Landsman () propose an approach for observational studies that is based on a population model and a treatment selection model and applying a combined likelihood to avoid strong ignorability assumptions.…”

Evaluating the Impact of Parent‐Reported Medical Home Status on Children's Health Care Utilization, Expenditures, and Quality: A Difference‐in‐Differences Analysis with Causal Inference Methods

“…Survey weights were adjusted in all statistical analyses. We followed the approach by Pfeffermann and Landsman () to adjusting survey weights in IPW and DR. As suggested by Potter () and Yu (), we truncated extremely small and large weights at the 5th and 95th percentiles of the analysis sample, respectively. We also evaluated the design effects of the MEPS (i.e., inflation in variance estimates) when accounting for survey design in the final ATGT and ATLT estimates.…”