Online instruction is quickly gaining in importance in U.S. higher education, but little rigorous evidence exists as to its effect on student learning. We measure the effect on learning outcomes of a prototypical interactive learning online statistics course by randomly assigning students on six public university campuses to take the course in a hybrid format (with machine‐guided instruction accompanied by one hour of face‐to‐face instruction each week) or a traditional format (as it is usually offered by their campus, typically with about three hours of face‐to‐face instruction each week). We find that learning outcomes are essentially the same—that students in the hybrid format are not harmed by this mode of instruction in terms of pass rates, final exam scores, and performance on a standardized assessment of statistical literacy. We also conduct speculative cost simulations and find that adopting hybrid models of instruction in large introductory courses has the potential to significantly reduce instructor compensation costs in the long run.

No abstract

, and seminar participants at Harvard/MIT for helpful comments and discussions, and Jeremy Ferwerda for developing estrat (available at SSC), a Stata package that calculates the leave-one-out and repeated split sample endogenous stratification estimators considered in this study. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.NBER working papers are circulated for discussion and comment purposes. They have not been peerreviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications.© 2013 by Alberto Abadie, Matthew M. Chingos, and Martin R. West. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source. Endogenous Stratification in Randomized Experiments ABSTRACTResearchers and policy makers are often interested in estimating how treatments or policy interventions affect the outcomes of those most in need of help. This concern has motivated the increasingly common practice of disaggregating experimental data by groups constructed on the basis of an index of baseline characteristics that predicts the values that individual outcomes would take on in the absence of the treatment. This article shows that substantial biases may arise in practice if the index is estimated, as is often the case, by regressing the outcome variable on baseline characteristics for the full sample of experimental controls. We analyze the behavior of leave-one-out and repeated split sample estimators and show they behave well in realistic scenarios, correcting the large bias problem of the full sample estimator. We use data from the National JTPA Study and the Tennessee STAR experiment to demonstrate the performance of alternative estimators and the magnitude of their biases.

No abstract