Bridging the Gap Between Research and Practice: Predicting What Will Work Locally

Abstract

This article addresses the gap between what works in research and what works in practice. Currently, research in evidence-based education policy and practice focuses on randomized controlled trials. These can support causal ascriptions (“It worked”) but provide little basis for local effectiveness predictions (“It will work here”), which are what matter for practice. We argue that moving from ascription to prediction by way of causal generalization (“It works”) is unrealistic and urge focusing research efforts directly on how to build local effectiveness predictions. We outline various kinds of information that can improve predictions and encourage using methods better equipped for acquiring that information. We compare our proposal with others advocating a better mix of methods, like implementation science, improvement science, and practice-based evidence.

Research Worth Using: (Re)Framing Research Evidence Quality for Educational Policymaking and Practice

“…As Wiliam (2019) succinctly stated, “Research tells us only what was, not what might be” (p. 132), highlighting the tension between “what worked” and “what will work.” 6 Yet the premise of evidence-based practice demands extrapolating from past inferences to make predictions about future contexts, 7 whether framed as universal generalizability (D. T. Campbell & Stanley, 1963) or as contextualized transferability or applicability (Cartwright & Hardie, 2017; Joyce & Cartwright, 2019; Lincoln & Guba, 1985; Miles & Huberman, 1994). Since people more readily transfer knowledge across similar contexts (e.g., Barnett & Ceci, 2002; Detterman, 1993), what deserves further study is determining the extent of similarity necessary for a given finding to successfully transfer in both conceptual use—where educators recognize the study setting as resembling their context—and instrumental use—where core features are sufficiently similar to justify legitimate applicability.…”

“… 11 Structures may be social, economic, cultural, or physical, such as sociocultural contexts, family dynamics, community norms, school culture, and organizational factors (Joyce & Cartwright, 2019; Phillips, 2019). Supports or drivers may include factors such as human capital, technology resources, or computer literacy (Joyce & Cartwright, 2019). Derailers or restrainers may include interruptions in a necessary resource, offsetting due to other factors undermining the effects, and self-defeat by causing other harms (Cartwright & Hardie, 2017).…”

Absorptive Capacity as a Means of Understanding and Addressing the Disconnects Between Research and Practice

“…Research conducted using in-depth study in similar contexts yields more usable information for practitioners. Practitioners value the results of research conducted in contexts that are similar with regard to variables such as school/district performance, urbanicity, student demographics, and teacher quality (Farley-Ripple et al, 2018; Joyce & Cartwright, 2019). They need research knowledge that is comprehensive and provides details of implementation and impacts; research evidence has to enable educators to meet a need for them to be motivated to use it (Ostinelli, 2016).…”

On the misinterpretation of effect size

“…Given the rarity with which we could ever conclude that one intervention is more effective than another on the basis of effect sizes from two different studies (or that one type of intervention is better than another type of intervention on the basis of effect size from two meta-analyses 2 ), it would also be a mistake to continue to propose a role for effect size in making decisions about policy and practice. Moreover, even if we could deal with those concerns, we would still have the difficulties associated with transporting results from the study context to policy and practice contexts (see Joyce & Cartwright, 2019).…”