Evaluative conversations: Translating between diverse stakeholders in regional RRI projects

Abstract

Since the summer of 2020, researchers from ten projects pertaining to the Horizon2020 Science with and for Society (SwafS) call have been meeting virtually as the SwafS14 Monitoring and Evaluation ecosystem. Topics of discussion were the trials and tribulations of their regional Responsible Research and Innovation (RRI) projects as well as their strategies for monitoring and evaluation. In this paper we make a first attempt at presenting these issues as problems of translation between different kinds of stakeholders. After an exploration of the diversity of stakeholders and the process of translation in regional RRI, we suggest evaluative conversations as a way of improving regional RRI. We intend to develop this idea in the future and that these conversations will facilitate more responsible and engaged monitoring and evaluation and contribute to better R&I policies.

Monitoring Open Science as transformative change: Towards a systemic framework

“…There is a long research experience of conducting surveys in order to understand the knowledge exchange practices in relation to a variety of collaborative practices (Díaz-Faes et al, 2023), modes of knowledge exchange with industry (D'Este & Patel, 2007), exchange with policy (Thune et al, 2023), innovation practices in industry through the Oslo Manual (Bloch, 2007;Gault, 2020;OECD & Eurostat, 2018), innovation in the public sector (Arundel et al, 2019), and practices of Responsible Research and Innovation (RRI) (Holtrop et al, 2022). Some studies aimed at monitoring OS processes also relied on surveys: for example on sharing of research materials (Shibayama et al, 2012), open peer review (Ross-Hellauer et al, 2017), policy use (Cole et al, 2023), engagement with non-academics (Lawson et al, 2019), integrity (Schneider et al, 2023), or broader perceptions and habits of OS (Ollé et al, 2023).…”

Monitoring Open Science as Transformative Change: Towards a Systemic Framework

“…There is a long research experience of conducting surveys in order to understand the knowledge exchange practices in relation to a variety of collaborative practices (Díaz-Faes et al, 2023), modes of knowledge exchange with industry (D'Este & Patel, 2007), exchange with policy (Thune et al, 2023), innovation practices in industry through the Oslo Manual (Bloch, 2007;Gault, 2020;OECD & Eurostat, 2018), innovation in the public sector (Arundel et al, 2019), and practices of Responsible Research and Innovation (RRI) (Holtrop et al, 2022). Some studies aimed at monitoring OS processes also relied on surveys: for example on sharing of research materials (Shibayama et al, 2012), open peer review (Ross-Hellauer et al, 2017), policy use (Cole et al, 2023), engagement with non-academics (Lawson et al, 2019), integrity (Schneider et al, 2023), or broader perceptions and habits of OS (Ollé et al, 2023).…”

Actors' Engagement in Monitoring and Evaluation Mechanisms for Responsible Research and Innovation: an Explorative Study of the AHP Technique

“…To our knowledge, there is just one previous study reviewing M&E methods and tools of RRI by van de Poel (2020), and its focus is on identifying pitfalls in M&E of RRI. Recent literature points out the need for monitoring and evaluating RRI at the territorial level (Völker et al, 2023) and through engaging processes with stakeholders that facilitate contextualized monitoring and evaluation of RRI (Holtrop et al, 2022). In the opposite direction, there are also calls for developing global indicators that overcome the contextualized approaches of M&E (Jensen, 2022).…”

“…Considering that M&E mechanisms of RRI are tools "entwined with the political and organizations context" in which they operate (Völker et al, 2023:05), we suggest opening up space for contestation and increasing social appraisal (Stirling, 2007). To this concern, implementing evaluative conversations since the early phases of the research process, as proposed by Holtrop et al (2022) and identifying structured approaches for the management of stakeholders' involvement and decisionmaking (Ten Holter, 2022) could reinforce the reflexivity and responsiveness of the research in this field.…”