Learning from the UK’s research impact assessment exercise: a case study of a retrospective impact assessment exercise and questions for the future

Abstract

National governments spend significant amounts of money supporting public research. However, in an era where the international economic climate has led to budget cuts, policymakers increasingly are looking to justify the returns from public investments, including in science and innovation. The so-called 'impact agenda' which has emerged in many countries around the world is part of this response; an attempt to understand and articulate for the public what benefits arise from the research that is funded. The United Kingdom is the most progressed in implementing this agenda and in 2014 the national research assessment exercise, the Research Excellence Framework, for the first time included the assessment of research impact as a component. For the first time within a dual funding system, funding would be awarded not only on the basis of the academic quality of research, but also on the wider impacts of that research. In this paper we outline the context and approach taken by the UK government, along with some of the core challenges that exist in implementing such an exercise. We then synthesise, together for the first time, the results of the only two national evaluations of the exercise and offer reflections for future exercises both in the UK and internationally.

Institutional hierarchies and research impact: new academic currencies, capital and position-taking in UK higher education

“…The UK context stands out in two ways pertinent to the discussion so far. First, for the strong prevalence of performance-based funding allocations (in the form of the REF), both as a general accountability mechanism and in advancing an impact agenda (Morgan Jones, Manville, and Chataway 2017, Marcella, Lockerbie, and Bloice 2016, Hill 2016, McKenna 2015, Nowotny, Scott, and Gibbons 2003, Hicks 2012)one of the most 'controversial' (S. Smith, Ward, andHouse 2011, 1369) aspects of research evaluation since impact became one of the three strands of assessment in REF2014 (alongside the outputs of research and the research environment). The second way in which the UK stands out is for its entrenched and much-discussed hierarchical structure in terms of differential prestige and power associated with different universities (Boliver 2015).…”

Research, discovery, and impact

“…This blind spot is garnering attention. Some governments are beginning to hold universities accountable for measuring and tracking the impact of taxpayer‐supported research, which has downstream implications for how faculty research efforts are evaluated 1 …”

Os desafios da sociedade não têm fronteiras: impacto da pesquisa acadêmica interdisciplinar em polí­ticas públicas

“…Aponta-se na literatura que o primeiro a expressar a ideia de que a pesquisa financiada com recursos públicos pode, e deve, ter um uso prático para a sociedade foi Vannevar Bush (1945) em seu tratado intitulado Science: The endless frontier (Lima & Wood Jr., 2014;Morgan Jones, Manville, & Chataway, 2017). Bush (1945) argumenta por um maior suporte governamental à ciência em áreas críticas para o interesse público (agricultura, habitação, saúde pública, etc.…”

“…A avaliação desses impactos foi realizada por meio da revisão de estudos de casos estruturados de até quatro páginas enviados pelas IES por parte dos comitês de cada grande área. Dois critérios foram considerados: importância (intensidade da influência ou efeito) e alcance (a propagação ou a amplitude de influência ou efeito sobre os grupos relevantes) (Hughes et al, 2019;Morgan Jones et al, 2017) Na primeira edição do REF, realizada em 2014 e referente ao período de 2008-2013, foram submetidos e avaliados 6.975 casos de impacto. Para a próxima avaliação, a ser realizada em 2021, o peso do impacto aumentará para 25% e o de resultados/outputs diminuirá para 60%.…”

“…Dessa maneira, um possível estudo relevante seria como a pesquisa acadêmica contribui para políticas públicas. No entanto, visto que a literatura da área tem apontado a pesquisa interdisciplinar como um tipo de pesquisa mais aderente à natureza dos problemas da sociedade (complexos e multifacetados), com os quais gestores de políticas públicas devem lidar em seu trabalho, optou-se por focar na pesquisa acadêmica interdisciplinar (Cooper, 2017;Lyall, Meagher, & Bruce, 2015;Morgan Jones, Manville, & Chataway, 2017). Além disso, agências de fomento têm dado especial atenção a esse tipo de pesquisa.…”