The unbearable lightness of scientometric indices

Migheli, Matteo; Ramello, Giovanni B.

doi:10.1002/mde.3486

Cited by 6 publications

(1 citation statement)

References 43 publications

(61 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ideally, if an article is accepted by a specific journal, it is because its overall quality is similar to other papers previously published in that journal [ 47 ]. However, journal-level indicators are, at best, only moderately suggestive of the quality of an article [ 48 ], which makes indicators that measure specific articles more popular [ 49 ].…”

Section: Resultsmentioning

confidence: 99%

Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation

Jiang,

Liu,

Zhang

et al. 2024

J Med Internet Res

View full text Add to dashboard Cite

Background As an important platform for researchers to present their academic findings, medical journals have a close relationship between their evaluation orientation and the value orientation of their published research results. However, the differences between the academic impact and level of disruptive innovation of medical journals have not been examined by any study yet. Objective This study aims to compare the relationships and differences between the academic impact, disruptive innovation levels, and peer review results of medical journals and published research papers. We also analyzed the similarities and differences in the impact evaluations, disruptive innovations, and peer reviews for different types of medical research papers and the underlying reasons. Methods The general and internal medicine Science Citation Index Expanded (SCIE) journals in 2018 were chosen as the study object to explore the differences in the academic impact and level of disruptive innovation of medical journals based on the OpenCitations Index of PubMed open PMID-to-PMID citations (POCI) and H1Connect databases, respectively, and we compared them with the results of peer review. Results First, the correlation coefficients of the Journal Disruption Index (JDI) with the Journal Cumulative Citation for 5 years (JCC5), Journal Impact Factor (JIF), and Journal Citation Indicator (JCI) were 0.677, 0.585, and 0.621, respectively. The correlation coefficient of the absolute disruption index (Dz) with the Cumulative Citation for 5 years (CC5) was 0.635. However, the average difference in the disruptive innovation and academic influence rankings of journals reached 20 places (about 17.5%). The average difference in the disruptive innovation and influence rankings of research papers reached about 2700 places (about 17.7%). The differences reflect the essential difference between the two evaluation systems. Second, the top 7 journals selected based on JDI, JCC5, JIF, and JCI were the same, and all of them were H-journals. Although 8 (8/15, 53%), 96 (96/150, 64%), and 880 (880/1500, 58.67%) of the top 0.1%, top 1%, and top 10% papers selected based on Dz and CC5, respectively, were the same. Third, research papers with the “changes clinical practice” tag showed only moderate innovation (4.96) and impact (241.67) levels but had high levels of peer-reviewed recognition (6.00) and attention (2.83). Conclusions The results of the study show that research evaluation based on innovative indicators is detached from the traditional impact evaluation system. The 3 evaluation systems (impact evaluation, disruptive innovation evaluation, and peer review) only have high consistency for authoritative journals and top papers. Neither a single impact indicator nor an innovative indicator can directly reflect the impact of medical research for clinical practice. How to establish an integrated, comprehensive, scientific, and reasonable journal evaluation system to improve the existing evaluation system of medical journals still needs further research.

show abstract

Section: Resultsmentioning

confidence: 99%

Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation

Jiang,

Liu,

Zhang

et al. 2024

J Med Internet Res

View full text Add to dashboard Cite

show abstract

Academic publication uncertainty and research behavior

Goel,

Faria

2024

Manage Decis Econ

View full text Add to dashboard Cite

There are many uncertainties in the production/dissemination of academic research. This paper incorporates publication uncertainty in a game model between researchers and journal editors, and examines its effects on the quantity and quality of research and on citations. Authors seek to maximize satisfaction from their publishing efforts, while journal editors try to enhance reputations of their journals. Publication probability depends on the number of academic journals. Results show that the impacts of a change in the number of journals (capturing publication uncertanty) on citations and quality are symmetric. Citations and research quality can go up, while total publications (research quantity) can increase, decrease, or stay unchanged.

show abstract

t-index: entropy based random document and citation analysis using average h-index

Singh

2022

Scientometrics

View full text Add to dashboard Cite

The unbearable lightness of scientometric indices

Cited by 6 publications

References 43 publications

Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation

Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation

Academic publication uncertainty and research behavior

t-index: entropy based random document and citation analysis using average h-index

Contact Info

Product

Resources

About