Interpreting Bibliometric Data

Abstract

Many academic analyses of good practice in the use of bibliometric data address only technical aspects and fail to account for and appreciate user requirements, expectations, and actual practice. Bibliometric indicators are rarely the only evidence put before any user group. In the present state of knowledge, it is more important to consider how quantitative evaluation can be made simple, transparent, and readily understood than it is to focus unduly on precision, accuracy, or scholarly notions of purity. We discuss how the interpretation of ‘performance’ from a presentation using accurate but summary bibliometrics can change when iterative deconstruction and visualization of the same dataset is applied. From the perspective of a research manager with limited resources, investment decisions can easily go awry at governmental, funding program, and institutional levels. By exploring select real-life data samples we also show how the specific composition of each dataset can influence interpretive outcomes.

A Decade of Post-Intensive Care Syndrome: A Bibliometric Network Analysis

“…We calculated each country's median Category Normalized Citation Impact (CNCI) [19] and relative research activity [20] for three time periods: 2012-2014, 2015-2017, and 2018-2021. The CNCI indicates the ratio of an article's citations and the average citations of articles within the same research field, document type, and year [19]. Hence, a CNCI >1 or <1 indicates above-average or below-average citations per article, respectively.…”

Research on Interlocking Directorates in Family Business: Evidence from Scopus Bibliometric Data

“…Results obtained on analysis of bibliometric data hardly address any meaningful objectives unless interpreted in the light of a clearly defined context [38]. In this study, we interpret the results of our analysis in light of the trend in directorate ties in family business research.…”

Highly Cited Papers at the Spanish Domestic Level

“…The analysis and evaluation of the research activity carried out in the institutions has been a decisive step to really know the scope of these activities, make proposals, and offer society the necessary transparency of its efficient management of the resources allocated to the research carried out in these institutions. In this way, Szomszor et al (2021) state that "Research evaluation may be seen as a reflection of a broader societal shift to institutional managerialism and public sector accountability." However, it is within higher education institutions where evaluation has been more ingrained and where it is playing a more decisive role.…”

“…The reasons why this effects have occurred are several, for example, accountability to society for the activities they carry out, the proper management of the financial resources they receive, or knowing how the scientific productivity of their academic staff evolves. One of the countries that first considered the need to evaluate its higher education institutions was the United Kingdom, where the first national Research Selectivity Exercise was introduced in 1986 and led to a more formalized and structured Research Assessment Exercise (RAE) from 1992 (Szomszor et al, 2021). This evaluation process has currently changed its name to Research Excellence Framework (REF) (REF, 2020) and it has had multiple counterparts in different countries (Sanz-Casado et al, 2013), especially in the Nordic countries (Sivertsen, 2018) and in Australia where the Australian Research Council (ARC) conducted the first Excellence in Research for Australia (ERA) evaluation in 2010 (ARC, 2019).…”