Can Online Off-the-Shelf Lessons Improve Student Outcomes? Evidence from a Field Experiment

Abstract

Many websites now warehouse instructional materials designed to be taught by teachers in a traditional classroom. What are the potential benefits of the new resources? We analyze an experiment in which we randomly give middle school math teachers access to existing high-quality, off-the-shelf lessons, and in some cases, support to promote their use. Teachers receiving access alone increased students’ math achievement by a marginally significant 0.06 of a standard deviation. Teachers who received access and support increased students’ math achievement by 0.09 of a standard deviation. Weaker teachers experience larger gains, suggesting that these lessons substitute for teacher skill or efforts. The online materials are more scalable and cost effective than most policies aimed at improving teacher quality, suggesting that, if search costs can be overcome, there is a real benefit to making high-quality instructional materials available to teachers on the Internet. (JEL C93, I21, J24, J45)

Special Educators’ Working Conditions, Self-Efficacy, and Practices Use With Students With Emotional/Behavioral Disorders

“…Logistical resources are practical supports that teachers need to fulfill their responsibilities, including curricular resources and planning time (O'Brien et al, 2019). A strong body of research with general educators, including studies permitting causal inferences, indicates curricular resources can powerfully shape the quality and effectiveness of teachers' instructional practices (e.g., Jackson & Makarin, 2018;Jimenez et al, 2014). A small body of qualitative and mixed-methods research with SETs affirms studies of general educators, suggesting SETs use stronger instructional practices when they have structured curricula that support them in planning and providing instruction relevant to their students' needs (Bishop et al, 2010;Brownell et al, 2014;Siuty et al, 2018).…”

Strengthening STEM Instruction in Schools: Learning From Research

“…Other nudges involve reminders, but the data are sparse. Webinars and an online discussion board for teachers did not significantly increase the effectiveness of a math curriculum intervention (Jackson & Makarin, 2018). On the contrary, in the arena of parenting, another highly complex practice, simple text message reminders to engage in literacy activities, such as letter identification, can effectively alter parenting practices and improve student achievement (York, Loeb, & Doss, 2018).…”

Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not

“…The screens allowed teachers to learn from student reactions and model the effective teaching style during the non-video portion of their lessons, potentially increasing student interest in a way that watching the videos as a solitary endeavor did not. Previous work has emphasized the importance of teachers being able to observe and learn form high quality teaching (Jackson and Bruegmann 2009;Jackson et al 2014;Papay et al 2020) and having an impetus to use new teaching techniques (Jackson and Makarin 2018) in improving student test scores, two indirect aspects in Classrooms but not Tablets.…”

“…Variability in teacher capacity and performance can be a substantial barrier, leading to inadequate learning even for those who are enrolled in school (Andrabi et al 2007;Muralidharan 2013;Jackson, Rockoff, and Staiger 2014;Bold et al 2019). Despite hundreds of billions of dollars being spent annually on teacher compensation and teachers' strong impact on student learning, very little is known about how to increase teacher effectiveness (Jackson, Rockoff, and Staiger 2014;Jackson and Makarin 2018;Papay et al 2020). In response to weak incentive structures, frequent teacher absenteeism, and a legacy of teacher-centered lecturing, policies in lower income countries to remedy teacher deficiencies have primarily taken one of two approaches:…”

“…Existing literature provides insight into why using education technology to combine an outside expert with the existing teacher as in the Classrooms intervention could be effective, yet it has not been explicitly tested. The ability to project the videos to the class could have acted somewhat like a commitment device, ensuring that the teachers viewed the full videos themselves, building on findings in the US that providing teachers support and impetus to use materials was more effective than providing materials alone (Jackson and Makarin 2018). During these viewings teachers could see student reactions to the content and the more engaging teaching style of the presenter on the screen-regularly displaying visual aids and key words, posing questions-and model their teaching practice after the most effective elements, almost like having a highly effective colleague to observe, one of the three pillars on which rapidly improving teaching quality rests (Jackson et al 2014).…”