Maximizing Research Use in the World We Actually Live In: Relationships, Organizations, and Interpretation

Abstract

When I started my job as research director of the Massachusetts Department of Elementary and Secondary Education twelve years ago, I thought my job was to figure out what worked. My agency was just beginning to have access to new and exciting longitudinal data on students and educators. I envisioned that I'd use those data along with strong designs for causal inference to determine which programs and policies were working and which were not. Once we knew those answers, I figured, we would get better policy that would improve outcomes for Massachusetts' students.But in my twelve years in this job, I've learned that the process of improving policy 1 through research is much subtler and more complex than I had initially imagined. Research influences policy more often than much of the academic community thinks, and more frequently every day as we learn how to do this work better. But its influence is less linear than researchers expect, and it is driven as much by relationships and organizational capacity as by the actual information studies produce. Research use operates through conversations, not code; structures in organizations, not standard errors; relationships, not randomized controlled trials.I worry that the growing national efforts to connect research and policy too frequently start from the same "find what works" frame of mind that I did twelve years ago. The "find what works" approach misunderstands the problem of research use as one of lack of information-either lack of information about the impact of a policy or lack of awareness by the policy maker about the available information-and a need for "translation" across sectors (Penuel et al. 2015). This belies the research literature about how research actually plays into the policy decision process. If 1. This essay centers on the influence of research on policy, rather than practice. This is both because I have more expertise in the policy process than I do in issues of direct practice and because AEFP members' research tends to focus more on policy. I suspect, however, that many of the same insights would also apply in practice settings.

Assessing the Quality of Education Research Through Its Relevance to Practice: An Integrative Review of Research-Practice Partnerships

“…The population of interest in this chapter are school and district leaders or local education leaders. In the United States, even though multiple actors are often responsible for decisions regarding education policy and implementation (Conaway, 2020; DeBray et al, 2014), contemporary educational policymaking remains a largely local affair as principals and district central office leaders play a central role as the primary decision makers on the programs and reforms that schools implement (Penuel et al, 2017). Research use may also vary among local education leaders by their professional roles within districts (Penuel et al, 2017).…”

“…Research use in policy decision-making is better characterized as a labyrinth process rather than a linear moment (Tseng & Nutley, 2014). Decision-making is an interactive process involving multiple people based on formal and informal interactions rather than single events, and research findings must be interpreted by practitioners to foster understanding that drives decisions (Conaway, 2020; Weiss, 1980, 1982). The use of research is not simply to be an access to information problem.…”

Communicating Intelligence Research

“…The public can broadly be considered an amalgamation of different publics, people like policymakers, practitioners, current and potential students, and general readers ( Conaway 2019 ; Tseng 2012 ). These are people who largely have no scientific training in any specific discipline and thus know very little about the scientific body of evidence that has accumulated in any field, including the field of intelligence research.…”

“… Tseng ( 2012 ) goes further to argue that the challenges for research to impact policy and the public go well beyond communication and dissemination but are also about who is doing the translation and how productive contexts for translation can be developed. For example, in education policy, a research-practice partnership ( Conaway 2019 ) is a form of long-term agreement where researchers work alongside practitioners who directly work with schools to initiate research questions that will have a higher likelihood of being useful both in terms of solving problems of practice and also making scientific advances.…”

“…Communication requires all parties involved to actively want to do so. It requires a genuine exchange, acceptance, and integration of information ( Conaway 2019 ). As Tseng ( 2012, p. 12 ) explains regarding the uses of research in policy and practice, “we do not simply face a communications problem of better conveying research; nor is it merely a dissemination problem of better distributing research.…”