Environmental policy evaluation in the EU: between learning, accountability, and political opportunities?

Schoenefeld, Jonas J.; Jordan, Andrew

doi:10.1080/09644016.2019.1549782

Cited by 47 publications

(22 citation statements)

References 69 publications

(83 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Making use of evaluation results is paramount in evaluation theory [12,18,50]; however, the drivers for conducting an evaluation in terms of its utilization vary, and span accountability, learning and political nature [12,14]. Ideally, an evaluation should capture and address the concerns of various stakeholders-not only the decision makers-and this calls for an elaborate evaluation design that is able to heed multiple kinds of uses [20].…”

Section: Use Of Evaluationsmentioning

confidence: 99%

“…We apply Vedung's [12] definition of evaluation as a 'careful retrospective assessment of the merit, worth and value of administration, output and outcome of government interventions, which is intended to play a role in future, practical action situations'. Moreover, we extend the boundaries of this definition to not only include retrospective ex-post evaluations but also prospective ex-ante assessments, since both types are recognized in the European Commission's [13] guidelines for better regulation, as well as in the European Union's (EU) currently enforced 7th Environment Action Programme as means to 'improve environmental integration and policy coherence' (see also [14]).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transition governance for energy efficiency - insights from a systematic review of Swedish policy evaluation practices

2019

View full text Add to dashboard Cite

Background: The transition towards a more sustainable energy system is urgent for addressing global environmental and social challenges, and will require transformative changes including improved energy efficiency in the built environment. To reach identified efficiency potentials, various policy instruments have been introduced but their effects are often unclear. In this paper, we argue that the outline of transformative policy strategies will require well-designed evaluations. The objective is to present a theory-based evaluation framework that can be used to assess existing evaluations, in order to support transformative policy strategies. The framework is also applied to provide insights from current Swedish evaluation practices. Methods: The theory-based evaluation framework presented builds on evaluation theory, policy analysis and transition research and is arranged around methods, value judgements and use of the evaluations. Moreover, key aspects from transition research are included to provide guidance for transformative efforts in the evaluations. The systematic review presented in the paper is qualitative, covering 33 policy evaluations for energy efficiency in buildings in Sweden, commissioned by Swedish governmental authorities over a decade. Results: The results of the review reveal a wide range of evaluations undertaken, using a sound methodological evaluation base that builds on a variety of methods for analysis, and application of multi-criteria analyses. Commonly, however, a rather narrow scope was applied and we note a missed opportunity for triangulation of findings. Key aspects for capturing transformative efforts, such as system-, scale-and multi-actor approaches, as well as visioning, experimentation and learning, were considered to varying extents, but could be more explicit and elaborate. Conclusion: In all, we find the proposed theory-based evaluation framework useful for assessing and discussing both robustness and transformative efforts of current policy and evaluation practices. The review of the Swedish policy evaluations further indicates sound evaluation practices, and a foundational structure for identifying and analysing transformative efforts. To fully support transformative changes, we suggest a wider system perspective and a more thorough multi-actor approach and actor involvement in the evaluations. We also stress the need to further link evaluation theory with transition research to design evaluations that can support transformative changes in society.

show abstract

Section: Use Of Evaluationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Transition governance for energy efficiency - insights from a systematic review of Swedish policy evaluation practices

2019

View full text Add to dashboard Cite

show abstract

“…As a practice, policy evaluation was initially rather unsystematic and mostly ad hoc. In recent decades, policy evaluation has also experienced an ongoing process of institutionalization [53,54]. To reduce the degree of subjectivity in the policy analysis process, evaluation requires systematic methodologies based on a set of normative criteria.…”

Section: How To Evaluate the Sustainability Paradigm?mentioning

confidence: 99%

“…The "side-effects evaluation" model [72] initially divides the effects of the policies under evaluation into anticipated and unanticipated effects occurring inside or outside the target area, and concludes with a qualitative categorization of effects. As Mickwitz [53] observes, unanticipated effects are often only partially known before an evaluation is actually undertaken. Therefore, sometimes the most important task of an evaluation is to uncover previously unknown unanticipated effects.…”

Section: How To Evaluate the Sustainability Paradigm?mentioning

confidence: 99%

Sustainability Science: A Paradigm in Crisis?

González-Márquez

Toledo

2020

Sustainability

View full text Add to dashboard Cite

The global socio-environmental crisis poses what is arguably the greatest challenge in the history of science. It has demanded an interdisciplinary effort in which thousands of scientists from around the world have rapidly articulated what is already recognized as a consolidated research field: Sustainability science [SS]. Considering the urgency of the matter, it is necessary to evaluate the progress so far achieved. How should this evaluation be carried out? This paper addresses this question taking into consideration some insights from the philosophy of science. In particular, it applies the conceptual framework developed by Thomas Kuhn to the study of scientific paradigms. It first reviews the development of SS, demonstrating that Kuhn’s model is followed step by step. The notion of problem-solving power is discussed as the main criterion for an evaluation of scientific paradigms. Next, several elements are presented suggesting that there is a general insufficiency of problem-solving power in SS. Furthermore, additional empirical data are considered early signs of a paradigm crisis. Subsequently, the way forward for SS is discussed. From Kuhn’s perspective, scientific progress is not only achieved by a steady accumulation of knowledge, but also by episodes of crises that precede radical qualitative leaps in which basic premises are modified. This paper concludes that the urgently needed progress in SS requires engaging in a critical revision of the fundamental claims upon which the field was constructed.

show abstract

“…The quality of policy monitoring data also relates to its relevance for the policy at hand. 5 What quality is (and for whom) will likely remain contested because it is intimately linked with various potential purposes of policy monitoring and evaluation, including accountability and learning functions, as well as politics (Schoenefeld and Jordan 2019;Schwartz and Mayne 2005). A policy monitoring system may originally be designed for checking policy goal attainment, but once the system starts producing information, this information may be used in many different ways.…”

Section: Qualitymentioning

confidence: 99%

Policy Monitoring in the EU: The Impact of Institutions, Implementation, and Quality

Schoenefeld¹,

Schulze²,

Hildén³

et al. 2019

Polit Vierteljahresschr

Self Cite

View full text Add to dashboard Cite

Policy monitoring is often seen as a crucial ingredient of policy evaluation, but theoretically informed empirical analyses of real-world policy monitoring practices are still rare. This paper addresses this gap by focusing on climate policy monitoring in the European Union, which has a relatively stringent system of greenhouse gas monitoring but a much less demanding approach to monitoring policies. It explores how institutional settings, policy implementation, and the quality of information may impact the practices and politics of policy monitoring. Drawing on quantitative regression models and qualitative interviews, it demonstrates that policy monitoring has evolved over time and is itself subject to implementation pressures, but also exhibits learning effects that improve its quality. In further developing both everyday policy monitoring practices and academic understanding of them, there is a need to pay attention to their design-specifically, the impact of any overarching rules, the institutional support for implementation, and the criteria governing the quality of the information they deliver. In short, policy monitoring should be treated as a governance activity in its own right, raising many different design challenges.

show abstract

Environmental policy evaluation in the EU: between learning, accountability, and political opportunities?

Cited by 47 publications

References 69 publications

Transition governance for energy efficiency - insights from a systematic review of Swedish policy evaluation practices

Transition governance for energy efficiency - insights from a systematic review of Swedish policy evaluation practices

Sustainability Science: A Paradigm in Crisis?

Policy Monitoring in the EU: The Impact of Institutions, Implementation, and Quality

Contact Info

Product

Resources

About