How Should Educational Effects Be Communicated to Teachers?

Abstract

Research findings regarding the effects of educational interventions—typically reported in units of standard deviations (e.g., Cohen’s d)—are often translated into more intuitive metrics before being communicated to teachers. However, there is no consensus about the most suitable metric, and no study has systematically examined how teachers respond to the different options. We conducted two preregistered studies addressing this issue. We found that teachers have strong preferences concerning effect size metrics in terms of informativeness, understandability, and helpfulness. These preferences challenge current research reporting recommendations. Most importantly, we found that different metrics induce different perceptions of an intervention’s effectiveness—a situation that could cause teachers to have unrealistic expectations about what a given intervention may achieve. Implications for how educational effects should be communicated are discussed.

A QuantCrit investigation of society’s educational debts due to racism, sexism, and classism in biology student learning

“…To contextualize the magnitude of the educational debts we found in our analyses, we follow the advice of Lortie-Forgues, Sio, and Inglis (2021) and discuss them in terms of the equivalent instruction time they represent. We use the overall average increase in scores from pretest to posttest of 16.3 percentage points as a proxy for the learning that occurs in a term.…”

Empirical Benchmarks to Interpret Intervention Effects on Student Achievement in Elementary and Secondary School: Meta-Analytic Results from Germany

“…For example, they are used by the What Works Clearinghouse (U.S. Department of Education et al, 2020b) as the so-called "improvement index" to assess the meaningfulness of intervention effects. However, it seems that teachers (at least in England) judge percentile gains to be less informative than other established effect size conversions (Lortie-Forgues et al, 2021). One reason may be that this methodin contrast to empirical benchmarksdoes not provide a familiar scale or comparison standard that puts percentile gains into a context that facilitates assessing their magnitude.…”

“…Years-of-learning is widely used because it provides an intuitive metric that can be easily communicated to researchers, practitioners, and policymakers. In particular, the study by Lortie-Forgues et al (2021) showed that teachers in England perceived converting intervention effects into years-of-learning to be an informative way to describe the impact of educational interventions, and to be more informative than other established effect size conversions (e.g., percentile gains).…”

What Do We Need to Teach New Teachers About Child Mental Health?

“…The global context of the purpose of education continues to be questioned, by not only those in the profession but by the general public (Bower, 2020). The health crisis, giving rise from Covid 19, has sparked many a debate and discussion interwoven with political ideology and produced an ongoing longing for evidence-based approaches, both within and outside educational spheres, to curricula and pedagogical effectiveness (Education Endowment Foundation (EEF), 2017; 2018); (Lortie-Forgues, 2021) and beyond to enquiries of whether mental health provisions are fit for purpose both in the wider community and within schools. It is strongly ascertained that the mental health crisis is apparent within the full scope of the population, affecting both children and adults with increasing economic cost.…”

“…The paper suggests that the mental health crisis impacting children has implications for new teachers in meeting the demand in the current global climate. (Education Endowment Foundation (EEF), 2017; 2018); (Lortie-Forgues, 2021). It explores why the definition of mental health might be a problematic construct for teachers and considers whether the teacher has a significant role to play in mental health provision in schools beyond the wider community.…”