Analysis of bibliometric indicators to determine citation bias

Abstract

Citations of research papers and citation-related indicators are frequently used factors in determining research priorities, allocating funding, and deciding appointments, promotions and tenures. The main problem with using citations for a variety of evaluations is a substantial difference in the average number of citations received by papers in different research fields and subfields. A large number of species are subjects of biological research, but the distributions of their citations have not been studied in detail. The key objective of the present work was to determine whether the choice of experimental subjects influence bias in citations. A case study was performed on papers from 108 plant species and five research fields. Funnel plot analyses and computer simulations identified species-related citation bias within all research fields. Relationships between bibliometric indicators imply that species with a fast growing number of publications in recent years (for example, new model organisms) generally have a higher average number of citations per paper than is the overall mean for the research field. In contrast, the average number of citations received by the five most prominent papers of the species was strongly correlated with the total number of published papers from laboratories working with the species. The current study indicates that despite the high frequency of cross-species citations, citations of species show a pattern similar to separate subfields. Although these analyses were performed on plant research papers, the findings have relevance for other areas of research where experimental subjects tend to form separate subfields. To reliably compare citations across subfields, a new type of bibliometric indicator is needed. In the meantime committees evaluating quality of research should take into consideration that citations of papers within a research field might be biased due to the experimental subjects used in the studies.

On bibliometrics in academic promotions: a case study in computer science and engineering in Italy

“…Empirical studies that, similarly to our work, show substantial bibliometric differences between papers in different sub-areas of the same discipline have recently been presented, e.g., by Simko [2015] and by Gorraiz et al [2016] focusing on biology and geography, respectively. The results show very heterogeneous publication strategies and different bibliometrics indicators, even within the same discipline.…”

“…The results show very heterogeneous publication strategies and different bibliometrics indicators, even within the same discipline. For instance, the investigation in Simko [2015] appears to confirm the anecdotal evidence, well-known among plant researchers, that research performed on certain plant species is much more likely to be cited: this corresponds to an extremely fine granularity level. We finally notice that in the first rounds of the ASN (2012-2013) only one scientific discipline (experimental physics) was divided into two subareas to accommodate for the multimodality of the distribution, partitioning professors of this discipline into two separate sets 2 .…”

Bibliometric Analysis of Publications of Universiti Teknologi Malaysia

“…Another significant finding here is that despite having a lower number of authors and publications, Materials Sciences (MATE) has a higher number of citations in comparison to those of Computer Science (COMP) which has 36% higher This finding suggests the high potential of these four fields to obtain even more citations in the future, provided that the publication numbers of these fields can be further increased. Bear in mind that the number of citations could be increased by increasing the number of publications [13].…”

Online Visibility, Social Networks and Glamorous Scientific Publications

“…(Ravenscroft et al, 2017, p. 1). Thus, the measurement of the influence of these publications in the scientific arena is made through the verification of impact indicators, despite their limitations (Konkiel, 2016;Ivan, 2015;Gammelgaard, 2016;Herteliu, Ausloos, Ileanu, Rotundo, & Andrei, 2017;Fabry & Fischer, 2017;Lăzăroiu, 2017;Wesel, 2016;Ravenscroft et al, 2017), which is even more evident in the area of social sciences (regarding Sociology, cf. Hermanowicz, 2016;Bornmann, 2017;Franco-Lopez, Sanz-Valero, & Culebras, 2017).…”