Evaluation in the age of evidence-based policymaking: Promises, pitfalls and paths forward

Abstract

Evaluators have long sought a world in which our work makes a tangible difference to society, but that goal has often seemed out of reach. However, in recent years, advocates have proclaimed an era of evidence-based policymaking in which the What Works data generated by evaluations will be increasingly used to inform programme and policy choices. Four primary factors have been critical to the rise of this approach – attaining a critical mass of curated What Works’ evidence, growing interest among political leaders in considering this information when making choices, new budgetary mechanisms for using these data and new tools that facilitate rigorous outcome studies. However, the movement also faces critical challenges including the growing distrust of empirical data among some political factions, leaks in the evaluation pipeline that generates data to identify What Works and the replication failure of many evidence-based interventions. The evaluation field should support this movement through efforts to plug leaks in the evidence pipeline, stronger efforts to assess implementation challenges, training students in evidence-based approaches and assisting in outreach to policymakers.

Scoping recent investment in evaluation education in Australian universities

“…Evaluators and employers are driving the demand, and high demand requires increased offerings. Solutions to evaluation challenges identified by VanLandingham emphasise ‘training students in evidence-based approaches and assisting in outreach to policymakers’ (VanLandingham, 2020, p. 131). This approach will futureproof new evaluators to be agile with skillsets to adapt to an evidence-based policy environment.…”

Cultivating researcher-policymaker partnerships: A randomized controlled trial of a model for training public psychologists.

“…Furthermore, by bridging the research and policy communities, researchers may find that it shapes the way they think about policy implications in their work, increasing the likelihood that research itself becomes responsive to decisionmakers’ needs (Tseng, 2014). Further study of systemic incentive structures and ways to increase the sensitivity of research production to meet decisionmakers’ needs (VanLandingham, 2020) are needed if we are to grow public psychology.…”

“…A series of recent public crises have emphasized the importance of using scientific evidence in public decision making (e.g., opioid epidemic, coronavirus disease 2019 [COVID-19] pandemic, growing public recognition of systemic racism). The complex interrelated behavioral health implications of these and other ongoing societal problems illustrate the need for the field to continue building capacity to work with public officials during policy development, as well as advance a culture and science of public psychology (Heinowitz et al, 2012; National Research Council, 2012; VanLandingham, 2020). Despite these pressing needs, scholars rarely receive formal support for translating their insights for policy audiences (Maton, 2016).…”

“…The Scholar Strategy Network also engages researchers in state-level policy efforts and supports nonacademic communication efforts, policy trainings, and local chapters for researchers to affiliate regionally (Skocpol, 2014). The Pew Institute’s Results First provides support to state-level legislators, as well as a robust evidence-based clearinghouse that incorporates data and benefit-cost estimates to support policymakers in the budgeting and appropriations processes (VanLandingham, 2020; VanLandingham & Silloway, 2016). The RPC model works with both state and federal legislators by assessing their policy goals and matching them with researchers to partner on policy development efforts (Crowley et al, 2018; Scott et al, 2019).…”

Evidence-based Policy in India: Crossing the Long, Uphill Bridge

“…Unfortunately, every so often, evaluations tend not to make categorical attempts at studying implementation challenges and end up with limited uptake amongst policymakers or other decision-makers (VanLandingham, 2020). The question of the utility of a piece of research finding assumes even more significance in studies where intended audiences include locally entrenched—and often, consequently knowledgeable – administrators or policymakers (e.g., a state mission manager of a programme and a district magistrate).…”