The transition to remote learning in the context of coronavirus disease 2019 (COVID-19) might have led to dramatic setbacks in education. Taking advantage of the fact that São Paulo State featured in-person classes for most of the first school quarter of 2020 but not thereafter, we estimate the effects of remote learning in secondary education using a differences-in-differences strategy that contrasts variation in students’ outcomes across different school quarters, before and during the pandemic. We also estimate intention-to-treat effects of reopening schools in the pandemic through a triple-differences strategy, contrasting changes in educational outcomes across municipalities and grades that resumed in-person classes or not over the last school quarter in 2020. We find that, under remote learning, dropout risk increased by 365% while test scores decreased by 0.32 s.d., as if students had only learned 27.5% of the in-person equivalent. Partially resuming in-person classes increased test scores by 20% relative to the control group.

No abstract

Background: The transition to remote learning in the context of Covid-19 could lead to dramatic setbacks for school enrollment and learning outcomes, especially in developing countries – where a multiplicity of challenges, from limited connectivity to little support from parents, are bound to limit its effectiveness. To date, however, no study has rigorously documented the educational impacts of remote learning relative to in-person classes within primary and secondary education. Quantifying the extent of those losses, as well as the extent to which resuming in-person classes in the pandemic could at least partially offset them, is urgent, as governments worldwide struggle evaluating the trade-offs between the health and educational risks of reopening schools, with vaccination rates still dragging.Methods: Taking advantage of the fact that São Paulo featured in-person classes for the lion’s share of the first school quarter of 2020, but not thereafter, we estimate the effects of remote learning on secondary education, using a differences-in-differences strategy that contrasts variation in dropout risk and standardized test scores between the first and the last school quarters in 2020 to that in 2019, when all classes were in-person. We estimate heterogeneous effects by grade, student characteristics and school characteristics. We also estimate intention-to-treat (ITT) effects of reopening schools in the pandemic through a differences-in-differences strategy, contrasting differences between middle- and high-school students within municipalities that authorized in-person classes to partially return for the latter over the last quarter of 2020, to those within municipalities that did not.Findings: Dropout risk increased by 365% under remote learning. While risk increased with local disease activity, most of it can be attributed directly to the absence of in-person classes:we estimate that dropout risk increased by no less than 247% across the State, even at the low end of the distribution of per capita Covid-19 cases. Average standardized test scores decreased by 0.32 standard deviation, as if students had only learned 27.5% of the in-person equivalent under remote learning. Learning losses did not systematically increase with local disease activity, attesting that they are in fact the outcome of remote learning, rather than a consequence of other health or economic impacts of Covid-19. Authorizing schools to partially reopen for in-person classes increased high-school students’ test scores by 20% relative to the control group.Interpretation: Results show that the societal costs of keeping schools closed in the pandemic are very large. While the learning losses that we document are at least as large as those documented in developed countries on the aftermath of the first Covid-19 wave, the dramatic surge in dropout risk is unique to developing countries. Such massive impacts are likely to bring about long-lasting effects on employment, productivity, and poverty levels. Our findings highlight that reopening schools under safe protocols can prevent those costs from growing even larger. They also caution against recent enthusiasm for remote learning in primary and secondary education outside the context of Covid-19.Funding: Research funded by the Inter-American Development Bank (IADB) as part of a partnership between IADB and the São Paulo State Education Secretariat.