Causal illusions in the classroom: how the distribution of student outcomes can promote false instructional beliefs

Abstract

Teachers sometimes believe in the efficacy of instructional practices that have little empirical support. These beliefs have proven difficult to efface despite strong challenges to their evidentiary basis. Teachers typically develop causal beliefs about the efficacy of instructional practices by inferring their effect on students' academic performance. Here, we evaluate whether causal inferences about instructional practices are susceptible to an outcome density effect using a contingency learning task. In a series of six experiments, participants were ostensibly presented with students' assessment outcomes, some of whom had supposedly received teaching via a novel technique and some of whom supposedly received ordinary instruction. The distributions of the assessment outcomes was manipulated to either have frequent positive outcomes (high outcome density condition) or infrequent positive outcomes (low outcome density condition). For both continuous and categorical assessment outcomes, participants in the high outcome density condition rated the novel instructional technique as effective, despite the fact that it either had no effect or had a negative effect on outcomes, while the participants in the low outcome density condition did not. These results suggest that when base rates of performance are high, participants may be particularly susceptible to drawing inaccurate inferences about the efficacy of instructional practices.

Emotion regulation efficacy beliefs: The outsized impact of base rates.

“…Participants resided in either the United States or the United Kingdom and a balanced sample for gender was sought. As no previous studies have examined the outcome density effect in emotion regulation, we conducted a power analysis (power = .80, α = .05) using a typical effect size ( d = .70) drawn from previous studies that examined the outcome density effect (e.g., Double et al, 2020). Based on this power analysis, a sample size of 60 was recruited.…”

“…The current study examines a particular kind of causal illusion in efficacy beliefs about other-directed emotion regulation, the outcome density effect. The outcome density effect refers to the fact that when a desired outcome occurs frequently, people are more likely to believe that they are causing it to occur, even when they are not having any causal effect on the outcome (Double et al, 2020;Matute et al, 2015). This effect has been shown in a variety of settings and purportedly plays a role in many pseudoscientific beliefs (Matute et al, 2011(Matute et al, , 2015.…”

“…While in general people accurately infer the causal effects of their actions, several biases have been noted previously. The outcome density effect, whereby one is prone to believe they are causing a desired outcome if it occurs frequently, has been shown in numerous settings and has been proposed to explain many inaccurate real-world beliefs (Double et al, 2020;Matute et al, 2011Matute et al, , 2015. The current experiments showed that the outcome density effect can be applied to the emotion regulation context-when the target of emotion regulation typically "feels good" (i.e., they have a high average affect), participants erroneously assumed that they were effective at regulating the target's emotions.…”

Instruction on the Scientific Method Provides (Some) Protection Against Illusions of Causality

“…where the potential cause is either the participant's own actions (e.g., Alloy & Abramson, 1979;Blanco & Matute, 2015), or a cue presented within a fictitious causal context such as a medical treatment scenario (e.g., Blanco et al, 2013). The illusion of causality and OD effects have been observed with outcomes presented as discrete events (i.e., "patient recovers" versus "patient does not recover") and with different distributions of continuous outcomes (Chow et al, 2019;Double et al, 2020). The influence of a high outcome probability can also be found in real-world health beliefs, where treatments that are not scientifically validated to be efficacious are commonly used for the treatment of illnesses that have a high rate of spontaneous remission akin to a high base-rate of recovery (e.g., Echinacea use and the common cold, Barrett et al, 2010;Karsch-Völk et al, 2014; see also Blanco & Matute 2019).…”

Establishing the adult learning sciences through a shared scientific attitude: A vision for the Future

“…Educators are not immune from similar misunderstandings. Double et al (2020) recently illustrated how educators form causal beliefs about the efficacy of their practices by inferring their effect on learners' performance. When learner performance is already high, educators are particularly susceptible to attributing it to the efficacy of their practice rather than the influence of other things.…”