Adding Fuel to the Flames? Politicisation of EU Policy Evaluation in National Parliaments

Abstract

Under which conditions does national parliamentary discourse politicise European Union (EU) policy evaluation? In times of multiple crises and uncertainty, the alleged 'democratic deficit' of the EU, defined as an apparent lack of legitimacy, has regained scholarly and popular attention. The European Commission and academic commentators consider policy evaluation a specialised and targeted tool to improve the 'output legitimacy' of the EU by assessing policy effectiveness and efficiency. While evaluation can strengthen output legitimacy directly via learning, evaluation can be particularly effective when it becomes part of the national communicative discourse on the EU. This discourse is most likely to take place in national parliaments, as they are the forums in which the government can be held to account. This paper relies on an automated content analysis of the share of keywords related to EU policy evaluation in debates in six national parliaments, covering a period of 20 years. The findings show that the combination of popular and party Euroscepticism is crucial in determining parliamentary debate on EU policy evaluation. Pro-European parties generally do not refer to policy evaluation. However, if political parties are critical of European integration, EU policy evaluation is mentioned more frequently. Under these conditions, members of parliaments also refer more frequently to EU policy evaluation as the public becomes more Eurosceptic. These findings suggest that EU policy evaluation is used as a tool for domestic political contestation, with potential negative normative implications for the output legitimacy of the EU and for evaluation as a tool for learning. Zusammenfassung Unter welchen Bedingungen wird die EU-Policy-Evaluation Gegenstand von politischen Debatten in nationalen Parlamenten der EU-Mitgliedstaaten? Vor dem Hintergrund des breit diskutierten Demokratiedefizits in der Europäischen Union wird die professionelle Evaluation von EU-Politiken sowohl von der Europäischen Kommission selbst als auch von akademischen Beobachtern als ein mögliches Instrument zur Verbesserung der "output legitimacy" der EU angesehen. Evaluation kann zur direkten Verbesserung von Fachpolitiken durch Lerneffekte führen, gleichzeitig aber auch indirekt die Qualität nationaler Diskurse in konkreten Politikfeldern steigern. Entsprechende Debatten sind insbesondere in den Plenen der nationalen Parlamente der EU-Mitgliedstaaten zu erwarten. Die vorliegende Studie analysiert mithilfe eines computergestützten Textanalyseverfahrens Plenardebatten in sechs Mitgliedstaaten über einen Zeitraum von 20 Jahren. Die Befunde zeigen, dass Euroskeptizismus, sowohl in der Bevölkerung als auch auf der Ebene der Parteien, die Häufigkeit der Nennung von Schlagwörtern mit Bezug zur Evaluation von EU-Politiken stark beeinflusst. Abgeordnete pro-europäischer Parteien politisieren Evaluation nur selten, während mehr über Evaluation diskutiert wird, wenn europaskeptische Parteien im Parlament vertreten sind. Insbesondere reagieren die Abgeordnet...

The Politicisation of Evaluation: Constructing and Contesting EU Policy Performance

“…Furthermore, a range of rapid technological developments, including digitalisation, big data and artificial intelligence are likely to become new opportunities for evaluators in terms of assessing policy impact, as well as for those seeking to understand politicisation. Hoerner's (2019) contribution in this special issue demonstrates the valuable insights that automated, large-scale quantiative analyses can deliver, and future scholars should extend such approaches to other areas of social life beyond formal political institutions, such as social media.…”

“…Furthermore, the more Eurosceptic the environment, the more evaluations were cited. According to Hoerner (2019), this suggests a high level of politicisation of evaluations. These findings chime with those of Dahler-Larsen and Sundby (2019), who similarly suggest that low evaluability can be a tool to drive certain types of (non-)use of evaluation.…”

Talking about Europe? Explaining the salience of the European Union in the plenaries of 17 national parliaments during 2006–2019

“…Research shows that the strategic political incentives of MPs and parties determine whether parliaments use their opportunity structures to communicate EU affairs (Auel et al, 2015, 2016; Auel and Raunio, 2014; Hoerner, 2019; Rauh and De Wilde, 2018). These two vote-seeking actors have scarce resources, requiring good reasons to engage with often low-salience and opaque EU issues (Miklin, 2014: 82).…”

“…First, internal party dissent on European integration may incentivize party leaders to keep the EU off the agenda to avoid electoral backlash by hiding internal conflict (Auel and Raunio, 2014). Second, I control for the level of parliamentary Euroscepticism as mainstream parties commonly hold more favorable views toward European integration than the general public, incentivizing Eurosceptic parties to compete on EU issues (Auel and Raunio, 2014; Hoerner, 2019).…”

“…The resulting EU salience score is a proportion and naturally skewed toward zero as EU matters compete with every other imaginable issue. Therefore, the variable was log-transformed (see Hoerner, 2019; Rauh and de Wilde, 2018).…”