<b>Implicated in the Indicator Game? An Experimental Debate</b>

Abstract

The rise of new modes evaluating academic work has substantially changed institutions and cultures of knowledge production. This has been reflected and criticized in the literature in STS and beyond. For STS scholars, these debates (should) however have an even more specific dimension. Many of us are experts on aspects of these changes. But at the same time, we too are part of the processes we are analyzing, and often criticizing. To put it slightly provocatively, often we cannot avoid playing the very same game that we scrutinize. This creates tensions that many of us reflect on, and it certainly has created many implicit and explicit normative stances on how to deal with them. Yet it seems that so far there has been little room in our field to reflect on and exchange this particular kind of experience-based knowledge. There are many different ways to engage with the dynamics of evaluation, measurement and competition in contemporary academia, or to play what we refer to colloquially here as the "indicator game." With this debate, we would like to give room to the expression and discussion of some of these ways. This text is the introduction and prompt to an experimental debate. We discuss the state of the academic discussion on the impact of indicator-based evaluation on academic organization, epistemic work and identities. We use insights from these debates to raise questions for how STS and STSers themselves deal with the indicator game. In conclusion, we summarize our contributors' arguments and propose the concept of "evaluative inquiry" as a new way of representing the quality of STS work in evaluative contexts.

Bridging the Evaluation Gap

“…We had hoped that this would be reciprocated by the evaluators entering our space and joining us in a common research enterprise. In other words, we wanted to turn the evaluation into what Fochler & de Rijcke (2017) call an "evaluative inquiry."! !…”

Implicated in the Indicator Game: A Response to the Debate

“…Still, the call, the workshop outlined in the introduction (Fochler and de Rijcke 2017) and this set of essays all articulate a strong sense that STS needs strategies in response to new modes of evaluation, measurement and competition that are restructuring academic institutions, practices, and lives. I fully endorse this, as well as appreciating that this can and will be approached in a variety of ways.…”

Gatekeepers of Reward: a Pilot Study on the Ethics of Editing and Competing Evaluations of Value

“…The studies conducted vary in their focal points, ranging from authorship cultures (Biagioli 2003;Marušić et al 2011;Youtie and Bozeman 2014), citation cultures (Shrum 2010;Wouters 1999), the use, abuse and performativity of metrics (Hicks et al 2015;Rushforth and de Rijcke 2015), the organisation, quality and consequences of peer review (Bornmann and Daniel 2010;Schwartzman 1997) to the online construction of the self by academics (Fochler and de Rijcke 2017;Hammarfelt et al 2016). Combined, they provide an ever-growing understanding of research practices and the attribution and evaluation of value in those practices.…”