Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

McKiernan, Erin C.; Schimanski, Lesley A.; Nieves, Carol Muñoz; Matthias, Lisa; Niles, Meredith T.; Alperín, Juan Pablo

doi:10.7554/elife.47338

Cited by 200 publications

(119 citation statements)

References 37 publications

(49 reference statements)

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…Institutions relying on traditional metrics, such as number of publications and associated journal impact factors may misinterpret what these metrics mean. 21 Beyond evidence, there are other reasons to consider alternative criteria. They may better align with a university's mission.…”

Section: Discussionmentioning

confidence: 99%

Academic criteria for promotion and tenure in faculties of biomedical sciences: a cross-sectional analysis of 146 universities

Rice

Raffoul

Ioannidis

et al. 2019

Preprint

View full text Add to dashboard Cite

Objectives:To determine the presence of a set of pre-specified traditional and progressive criteria used to assess scientists for promotion and tenure in faculties of biomedical sciences among universities worldwide.Design: Cross-sectional study.Setting: Not applicable.Participants: 170 randomly selected universities from the Leiden Ranking of world universities list were considered. Main outcome measures:Two independent reviewers searched for all guidelines applied when assessing scientists for promotion and tenure for institutions with biomedical faculties. Where faculty-level guidelines were not available, institution-level guidelines were sought. Available documents were reviewed and the presence of 5 traditional (e.g., number of publications) and 7 progressive (e.g., data sharing) criteria was noted in guidelines for assessing assistant professors, associate professors, professors, and the granting of tenure. Results:A total of 146 institutions had faculties of biomedical sciences with 92 having eligible guidelines available to review. Traditional criteria were more commonly reported than progressive criteria (t(82)= 15.1, p= .001). Traditional criteria mentioned peer-reviewed publications, authorship order, journal impact, grant funding, and national or international reputation in 95%, 37%, 28%, 67%, and 48% of the guidelines, respectively. Conversely, among progressive criteria only citations (any mention in 26%) and accommodations for extenuating circumstances (37%) were relatively commonly mentioned; while there was rare mention of alternative metrics for sharing research (2%) and data sharing (1%), and 3 criteria (publishing in open access mediums, registering research, and adhering to reporting guidelines) were not found in any institution

show abstract

Section: Discussionmentioning

confidence: 99%

Academic criteria for promotion and tenure in faculties of biomedical sciences: a cross-sectional analysis of 146 universities

Rice

Raffoul

Ioannidis

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…20 However, there is no perfect bibliometric indicator and we used it because the JIF is highly prevalent, and most researchers are familiar with this metric. 21 Also, we did not analyse all medical fields but focused on one field of interest. Our survey had a response rate of 17%, which carries a high risk of non-responder bias.…”

Section: Open Access Limitationsmentioning

confidence: 99%

Definitions of blinding in randomised controlled trials of interventions published in high-impact anaesthesiology journals: a methodological study and survey of authors

Penić¹,

Begic

Balajić

et al. 2020

BMJ Open

View full text Add to dashboard Cite

ObjectivesTo analyse the completeness of reporting of blinding in randomised controlled trials (RCTs) of interventions in anaesthesiology, the actual blinding status of various persons associated with an RCT and trial authors’ interpretation of blinding terminology related to RCTs.MethodsThis was a methodological study and a cross-sectional survey. We analysed reporting related to blinding in published RCTs of interventions published in seven highly cited anaesthesiology journals from 2014 to 2016 and registered protocols in ClinicalTrials.gov. We surveyed corresponding authors of included RCTs about their definitions of blinding. The primary outcome was the number of RCTs that explicitly described who was blinded in a trial. Secondary outcomes were definitions of blinding terminology in the trials; trial authors’ interpretation of blinding terminology; discrepancies in the blinding description within registered protocols and between registered protocols and publications.ResultsOut of 622 analysed RCTs, 38% were not explicitly described as either open label or blinded studies and 10% did not report any information about blinding or lack of blinding. Only one manuscript fully reported the status of blinding for various individuals that may be involved with a trial. The most common descriptor was that a trial was double-blind. We found discrepant information regarding blinding in the majority of registered protocols. Even when there were no discrepancies in the registration, we found discrepancies in the reporting of blinding between the majority of registered protocols and published manuscripts. The survey of authors (40 responses from 231 eligible authors; 17% response rate) of analysed RCTs showed that they differed in how they defined different levels of blinding in trials.ConclusionsReporting of the blinding status of key individuals involved in analysed anaesthesiology RCTs was insufficient. For reporting guidelines, peer reviewers and editors should insist on clear information on who was blinded in a trial instead of using the term ‘double-blind’ for different blinding practices.

show abstract

“…However, they are worthless if not getting accepted by the respective target groups. The analysis presented in McKiernan et al (2019) shows that a big part of universities and research institution rely on and trust in well-known metrics like the Journal Impact Factor (JIF) and use them in reviews, promotions, and tenure documents. It is improbable that they will supplant well-established metrics by novel BT-based approaches in the near future.…”

Section: Blockchain Technology As An Open Science Infrastructurementioning

confidence: 99%

A Review on Blockchain Technology and Blockchain Projects Fostering Open Science

et al. 2019

View full text Add to dashboard Cite

Many sectors, like finance, medicine, manufacturing, and education, use blockchain applications to profit from the unique bundle of characteristics of this technology. Blockchain technology (BT) promises benefits in trustability, collaboration, organization, identification, credibility, and transparency. In this paper, we conduct an analysis in which we show how open science can benefit from this technology and its properties. For this, we determined the requirements of an open science ecosystem and compared them with the characteristics of BT to prove that the technology suits as an infrastructure. We also review literature and promising blockchain-based projects for open science to describe the current research situation. To this end, we examine the projects in particular for their relevance and contribution to open science and categorize them afterwards according to their primary purpose. Several of them already provide functionalities that can have a positive impact on current research workflows. So, BT offers promising possibilities for its use in science, but why is it then not used on a large-scale in that area? To answer this question, we point out various shortcomings, challenges, unanswered questions, and research potentials that we found in the literature and identified during our analysis. These topics shall serve as starting points for future research to foster the BT for open science and beyond, especially in the long-term.

show abstract

Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

Cited by 200 publications

References 37 publications

Academic criteria for promotion and tenure in faculties of biomedical sciences: a cross-sectional analysis of 146 universities

Academic criteria for promotion and tenure in faculties of biomedical sciences: a cross-sectional analysis of 146 universities

Definitions of blinding in randomised controlled trials of interventions published in high-impact anaesthesiology journals: a methodological study and survey of authors

A Review on Blockchain Technology and Blockchain Projects Fostering Open Science

Contact Info

Product

Resources

About