Reconsidering Counting Articles in Ranked Venues (CARV) as the Appropriate Evaluation Criteria for the Advancement of Democratic Discourse in the IS Field

Debate Section Editorial Note: Information Systems Research: Thinking Outside the Basket and Beyond the Journal

“…Fitzgerald et al conclude that impact factors do not represent valid measures, that the basket of eight does not represent a reliable measure for quality, and that journals represent an unreliable measure for paper quality. Thus, they agree with Cuellar et al (2019) that one should not use the number of papers that a researcher has published in a certain journal for promotion and tenure assessments and argue that we should stop using journal impact factors and the journal basket. Instead, they propose that paper-level measures represent the best way forward if we continue to use metrics at all.…”

“…Like four of its predecessors, it deals with the way and where we publish and implicitly with the relationship between publication outlets and how we evaluate individual scholarly output for hiring, tenure, and promotion purposes. The previous debate, as the regular debate reader might recall, focused on "Reconsidering Counting Articles in Ranked Venues (CARV) as the Appropriate Evaluation Criteria for the Advancement of Democratic Discourse in the IS Field" (Cuellar, Truex, & Takeda 2019) and strongly emphasized the evaluation process. With their paper entitled "Information Systems Research: Thinking Outside the Basket and Beyond the Journal", Brian Fitzgerald, Alan R. Dennis, Juyoung An, Satoshi Tsutsui, Rishikesh C. Muchhala (2019) trigger the current debate.…”

Rejoinder to “Reconsidering Counting Articles in Ranked Venues (CARV) as the Appropriate Evaluation Criteria for the Advancement of Democratic Discourse in the IS Field”

“…We appreciate that Cuellar et al (2019) point to the weaknesses of the CARV approach. Scholars may feel more at ease articulating their doubts with respect to CARV as a result of their article being published in CAIS.…”

“…In this commentary, we respond to Cuellar, Truex, and Takeda (2019) who criticize the "process for evaluating scholarly output, counting articles in ranked venues (CARV)" (p. 188). In their view, CARV limits scholars from openly exchanging ideas, and thereby, democratic discourse.…”

“…2 Mingling the Effects of CARV (Quality, Growth, Author Evaluation) Cuellar et al (2019) attempt to integrate aspects of a free exchange of ideas as a means to obtain (or at least encourage) sought-after "output quality" and of growing the IS field on the one hand while pointing to inadequate author assessments that arise from applying CARV on the other. In contrast, we argue that assessing IS research quality, fostering the growth of the IS field, and evaluating IS academics in the context of promotion and tenure (P&T), hiring, or degree granting decisions conceptually differ and, thus, that one should not conflate them.…”