When Studying and Nudging Don’t Go as Planned: Unsuccessful Attempts to Help Traditional and Online College Students

Abstract

Previously circulated as "Lack of Study Time is the Problem, but What is the Solution? Unsuccessful Attempts to Help Traditional and Online College Students." At the University of Toronto, we are indebted to the first-year economics instructors for their willingness to incorporate an experiment into their courses for a fourth consecutive year. We especially thank James Lahey, our web developer, for his tireless commitment to designing and perfecting the experiment's website, as well as for his help with organizing and extracting the experimental data.

When Goal-Setting Forges Ahead but Stops Short

“…Similarly, Schippers et al (2015) find that goal-setting significantly reduces inequalities in achievement if implemented early in students' academic careers. 24 However, our findings are consistent with recent experimental studies on goal-settings and nudges that also do not find any impact on test scores (see Dobriyoni et al (2017); Oreopoulos et al (2018); and Oreopoulos & Petronijevic (2019)). Similar to Oreopoulos et al (2018), we find that both treatments led to a significant change in students' time use behavior, but this positive change did not translate into improvements in academic outcomes.…”

“…24 However, our findings are consistent with recent experimental studies on goal-settings and nudges that also do not find any impact on test scores (see Dobriyoni et al (2017); Oreopoulos et al (2018); and Oreopoulos & Petronijevic (2019)). Similar to Oreopoulos et al (2018), we find that both treatments led to a significant change in students' time use behavior, but this positive change did not translate into improvements in academic outcomes. An important difference is the the study by Oreopoulos et al (2018) was conducted in a developed country.…”

“…Similar to Oreopoulos et al (2018), we find that both treatments led to a significant change in students' time use behavior, but this positive change did not translate into improvements in academic outcomes. An important difference is the the study by Oreopoulos et al (2018) was conducted in a developed country. Also, our analysis of goal-setting reveals that a large fraction of students in the treatment schools had set very high goals compared to their baseline performance (less realistic goals), which may have resulted in sub-optimal levels of effort that do not align well with their ability.…”

Upgrading Education with Technology: Insights from Experimental Research

“…However, the study's sample size is very small and was selected from volunteers who responded to an advertisement-the students who might be most likely to enthusiastically make use of such an intervention-making it difficult to assess the external validity of these findings. Indeed, an attempt to replicate this online goal-setting exercise by one of this article's coauthors (Oreopoulos) found precise null effects for a sample of nearly 2,000 first-year university students, as well as for subgroups of students more at risk of poor academic performance (Oreopoulos, Patterson, et al 2018).…”

Growth-Mindset Interventions at Scale: Experimental Evidence From Argentina

“…This is certainly the case in developed countries like the United States, where it can be delivered online (see, for example, Gandhi et al, 2019; Paunesku et al, 2015; Yeager et al, 2016, 2019). The potential for deploying these interventions at a low cost is arguably even larger in higher education, where students already regularly interact with instructors online (Oreopoulos et al, 2017, 2018; Oreopoulos & Petronijevic, 2018).…”