The Evolution of Current Research Impact Metrics

Abstract

The prestige of publication has been based on traditional citation metrics, most commonly journal impact factor. However, the Internet has radically changed the speed, flow, and sharing of medical information. Furthermore, the explosion of social media, along with development of popular professional and scientific websites and blogs, has led to the need for alternative metrics, known as altmetrics, to quantify the wider impact of research. We explore the evolution of current research impact metrics and examine the evolving role of altmetrics in measuring the wider impact of research. We suggest that altmetrics used in research evaluation should be part of an informed peer-review process such as traditional metrics. Moreover, results based on altmetrics must not lead to direct decision making about research, but instead, should be used to assist experts in making decisions. Finally, traditional and alternative metrics should complement, not replace, each other in the peer-review process.

The Use of Twitter by the Trauma and Orthopaedic Surgery Journals: Twitter Activity, Impact Factor, and Alternative Metrics

“…Butler, et al proposed that alternative metrics are likely to play a significant role in literature evaluation going forward alongside traditional metrics [ 9 ]. However, the use of alternative metrics is not without limitations [ 9 ].…”

“…While a single research output may be present in one location online, the activity surrounding it may spread across a multitude of SoMe and online platforms [ 8 ]. Altmetrics evaluate the impact of these scholarly materials via online metrics, with an emphasis on data arising from social media outlets, including clicks, views, shares, downloads, saves, tweets, retweets, tags, mentions, bookmarks, commentary, as well as citations on Wikipedia [ 8 - 9 ]. This provides a complimentary online dimension to traditional citation metrics, i.e.…”

Comprehensive Researcher Achievement Model (CRAM): a framework for measuring researcher achievement, impact and influence derived from a systematic literature review of metrics and models

“…In contradistinction with the metrics discussed above, 54 papers (11.3%) discussed altmetrics (or ‘alternative metrics’), which included a wide range of techniques to measure non-traditional, non-citation based usage of articles, that is, influence 17. Altmetric measures included the number of online article views,93 bookmarks,94 downloads,41 PageRank algorithms95 and attention by mainstream news,63 in books96 and social media, for example, in blogs, commentaries, online topic reviews or Tweets 97 98. These metrics typically measure the ‘web visibility’ of an output 99.…”

Altmetrics: State of the Art and a Look into the Future

“…On the one hand, the "visibility and presence of altmetrics are quite impressive" [8] because they are used as marketing tools by many scientific publishers, more than 300 publications on the subject have appeared, and there are even conferences dedicated solely to altmetrics. On the other hand, there is no uniform definition, and therefore no consensus on what exactly is measured by altmetrics and what conclusions can be drawn from the results [8][9][10]. The only consensus regarding the term definition is that the indicators discussed are intended to measure the attention paid to scientific output where bibliometrics reaches its limits-that is, on the Internet [6].…”

“…In contrast, within the community itself, there is a general consensus that both disciplines complement each other instead of one excluding the other [11]. Altmetrics are not intended to replace the peer review process or bibliometrics; rather, they should be viewed as a second opinion [10] and a "new perspective on communication by and about science in social media" [7]. A report by the expert group on altmetrics on behalf of the European commission also argues for classical bibliometrics that they "offer complementary approaches to evaluation" together with alternative metrics [12].…”