A Theory of Quasi-Experimental Evaluation of School Quality

Abstract

Many centralized school admissions systems use lotteries to ration limited seats at oversubscribed schools. The resulting random assignment is used by empirical researchers to identify the effects of schools on outcomes like test scores. I first find that the two most popular empirical research designs may not successfully extract a random assignment of applicants to schools. When are the research designs able to overcome this problem? I show the following main results for a class of data-generating mechanisms containing those used in practice: The first-choice research design extracts a random assignment under a mechanism if the mechanism is strategy-proof for schools. In contrast, the other qualification instrument research design does not necessarily extract a random assignment under any mechanism. The former research design is therefore more compelling than the latter. Many applications of the two research designs need some implicit assumption, such as large-sample approximately random assignment, to justify their empirical strategy. This paper was accepted by Yan Chen, decision analysis.

Algorithm is Experiment: Machine Learning, Market Design, and Policy Eligibility Rules

“…It will also be challenging but interesting to develop finite-sample optimal estimation and inference strategies such as those recently introduced by Armstrong andKolesár (2018, 2020) and Imbens and Wager (2019). Finite-sample bias is also a related important topic for further work (Narita, 2020). Finally, we look forward to empirical applications of our method in a variety of business, policy, and scientific domains.…”