The constitutive effects of evaluation systems: Lessons from the policymaking process of Danish Active Labour Market Policies

Abstract

In recent years, evaluation research has taken an increasing interest in the proliferation of evaluation systems. One of the main assumptions being that such evaluation systems can enhance the use of evaluations and their findings. The current article analyses this assumption by empirically studying the workings of a specific evaluation system within the Danish Ministry of Employment. By applying the concept of constitutive effects, the article analyses how the evaluation system affects processes of policymaking within the central government. It is shown how the institutionalisation of an evidence-based evaluation system, at the ministerial level, structures the process of policymaking around a specific understanding of policy content, timeframes and expertise. Based on these findings, it is argued that evaluation systems can both enhance the instrumental use of evaluations and significantly alter policymaking processes.

Evaluating the evaluators: What have we learned from “neutral assessments” of the Canadian federal evaluation function?

“…This can be done by highlighting instances of instrumental or conceptual uses (Johnson et al 2009), which describe how evaluation results and recommendations were implemented to improve programs or to support organization-wide decision-making. Neutral assessments should also comment on whether evaluations contribute to the accumulated body of evidence or policymaking (Andersen 2019;Ginsburg and Rhett 2003;Olejniczak, Raimondo and Kupiec, 2016), and on the role that evaluators have played in fostering evaluation use (Olejniczak, Raimondo and Kupiec 2016). Such evidence would serve not only to demonstrate the departmental function's value internally, but could contribute to creating a body of evidence on governmental results over time, as was advocated by the evaluators of the 2009 Policy on Evaluation.…”

Using systematic reviews to inform environmental policy-making

“…Table 1 helps to understand how evaluations may be used in general, but not all are fully applicable to our case study. The three latter ways of use (tactical, ritual and constitutive) largely presuppose that evaluation activities are permanent and ongoing (Andersen, 2020), with institutionalised evaluation activities according to expected organisational needs and routines (Leeuw and Furubo, 2008). In our case, it was a temporary and independent research programme (EviEM) that decided whether to initiate each SR and on what topic.…”

The power of ideas in policymaking processes: The role of institutionalised knowledge production in state bureaucracies

“…In the last decade, the capacity for producing and mobilising knowledge within the Danish Ministry of Employment (MoE) has been altered significantly according to an ideal of evidence‐based policymaking (EBP). Based on a narrow definition of evidence‐based knowledge as knowledge derived from experimental or quasi‐experimental impact evaluations regarding the employment effects of different interventions (Andersen, 2020a), the MoE has developed routines, procedures and strategies for systematising and institutionalising the production, dissemination and use of this knowledge in policymaking processes. As elaborated in the following, the current capacity of the MoE for knowledge production thus resembles a highly institutionalised evaluation system on all of the aforementioned four criteria (Leeuw & Furubo, 2008).…”

“…1 1 The following is a brief introduction to the expansion and institutionalisation of the evidence‐based evaluation system. For a more thorough account, see Andersen (2020a, 2020b). …”