Assessing Scholarly Influence: Using the Hirsch Indices to Reframe the Discourse

Abstract

This study is part of a program aimed at creating measures enabling a fairer and more complete assessment of a scholar's contribution to a field, thus bringing greater rationality and transparency to the promotion and tenure process. It finds current approaches toward the evaluation of research productivity to be simplistic, atheoretic, and biased toward reinforcing existing reputation and power structures. This study examines the use of the Hirsch family of indices, a robust and theoretically informed metric, as an addition to prior approaches to assessing the scholarly influence of IS researchers. It finds that while the top tier journals are important indications of a scholar's impact, they are neither the only nor, indeed, the most important sources of scholarly influence. Other ranking studies, by narrowly bounding the venues included in those studies, distort the discourse and effectively privilege certain venues by declaring them to be more highly influential than warranted. The study identifies three different categories of scholars: those who publish primarily in North American journals, those who publish primarily in European journals, and a transnational set of authors who publish in both geographies. Excluding the transnational scholars, for the scholars who published in these journal sets during the period of this analysis, we find that North American scholars tend to be more influential than European scholars, on average. We attribute this difference to a difference in the publication culture of the different geographies. This study also suggests that the influence of authors who publish in the European journal set is concentrated at a moderate level of influence, while the influence of those who publish in the North American journal set is dispersed between those with high influence and those with relatively low influence. Therefore, to be a part of the top European scholar list requires a higher level of influence than to be a part of the top North American scholar list.

Publication Anxiety, Quality, and Journal Rankings: Researcher Views

“…Future studies can also investigate the question of the diversity of authorship and diversity of publications. Such evidence that we have suggests a predominance of North American authorship (Truex et al, 2009, Keller & Coulthard, 2013. Will this increase or decrease?…”

“…IS has been accused of having a North American bias (Gallivan & Benbunan-Fich, 2007) and that North American authors predominate in the top journals (Truex, Cuellar, & Takeda, 2009, Keller & Coulthard, 2013. Indeed, Tourish (2015) has identified that 31 out of the top 33 journals in the rankings produced by the Association of Business Schools in the UK were of US origin.…”

Lack of Gender Bias in Citation Rates of Publications by Dendrochronologists: What is Unique about this Discipline?

“…First-authorship is granted to the author who has made the largest intellectual contribution to that paper, so we examined only first-authored papers in this study. The fact that we worked with only first-authored papers also explains why we did not use the Hirsch ''H'' Index (Truex et al 2009), which is calculated from a researcher's entire body of publications, when we examined citation rates. The dendrochronologists examined in this study included academic and government researchers from 16 countries (Argentina, Australia, Canada, China, Ethiopia, Germany, Italy, Morocco, New Zealand, Russia, Slovenia, South Africa, Sweden, Switzerland, United Kingdom, and the United States of America) and spanned six continents.…”

Global Ranking of Management- and Clinical-Centered eHealth Journals

“…To avoid the "path dependency" problem (Truex, Cuellar, & Takeda, 2009), respondents could add and rank up to three additional journals for each group. We asked respondents to rank each journal's overall contribution on a seven-point Likerttype scale (0: none, 1: marginal, 2: some, 3: average, good: 4, very good: 5, outstanding: 6), and we aggregated the scores for each journal.…”