Epistemic Consequences of Bibliometrics-based Evaluation: Insights from the Scientific Community

Castellani, Tommaso; Pontecorvo, E.; Valente, Adriana

doi:10.1080/02691728.2015.1065929

Cited by 10 publications

(7 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, interdisciplinary journals present a mean rank lower than specialized, showing no evident advantage. This is in line with Castellani et al, (2016), which determined that those journals do not present a higher citation impact (with the exception of some journals like Nature). Moreover, as pointed Van Noorden (2015), interdisciplinary papers have a higher citation impact in the long-term.…”

Section: Regarding the Relationship With Bibliometric Indicatorssupporting

confidence: 88%

“…Hence, these papers could have a broad societal and economic impact, not captured by citations (Brown, 2018). However, one of the main challenges identified in this research is its difficulty to be published (Castellani et al, 2016). On the other hand, specialized journals are fundamental for achieving a deep scientific understanding and comprehension of specific research domain.…”

Section: Discussionmentioning

confidence: 99%

“…On the other hand, specialized journals are fundamental for achieving a deep scientific understanding and comprehension of specific research domain. In addition, it has linked in previous studies with a high impact factor (Castellani et al, 2016). Despite this, one of the problems of those journals is its limited audience as well as the lack of experts on the field.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Role taxonomy of green and sustainable science and technology journals: exportation, importation, specialization and interdisciplinarity

2021

View full text Add to dashboard Cite

Sustainability science is, per se, a topic that is inherently interdisciplinarity and oriented towards the resolution of societal problems. In this paper, we propose a classification of scientific journals that composes the journal category “Green and Sustainable Science and Technology” in the period 2014–2018 through the entropy-based disciplinarity indicator (EBDI). This indicator allows the classification of scientific journals in four types based on the citing and cited dimensions: knowledge importer, knowledge exporter, disciplinary and interdisciplinarity. Moreover, the relationship between this taxonomy and the JCR bibliometric indicators and its predictive capacity of the taxonomy is explored through a CHAID tree. As well, relations between the Web of Science categories, journals and taxonomy are explored by the co-occurrence of categories and correspondence analysis. Results suggest that the great majority of journals in this field are specialized or interdisciplinary. However, over the 5-year period proposed in this study, interdisciplinary journals tend to be far more stable than specialized ones. The decision tree has shown that the number of citations is the variable with the greatest discriminating capacity.

show abstract

Section: Regarding the Relationship With Bibliometric Indicatorssupporting

confidence: 88%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Role taxonomy of green and sustainable science and technology journals: exportation, importation, specialization and interdisciplinarity

2021

View full text Add to dashboard Cite

show abstract

“…Section 2.2 suggested the use of alternative nested H indexes based on linear and cubic fittings of standardized numbers of articles and citations, whereas Section 2.3 presented two-dimensional graphs based on alternative nested H and G indexes. These approaches would reduce incentives to engage in tactical or opportunistic behaviors in publication and citation by authors and journal editors [76][77][78][79][80][81][82], and should reduce discrimination against heterodox and interdisciplinary PHs that would be characterized by few citations and few articles [83,84]. Table 4 summarizes suggested warning symptoms that could be used to identify potentially questionable practices by editors and authors, although future experimental work based on analytical insights will be necessary to test whether these symptoms truly indicate manipulation of the PH quality.…”

Section: Discussionmentioning

confidence: 99%

Scientific Production and Productivity for Characterizing an Author’s Publication History: Simple and Nested Gini’s and Hirsch’s Indexes Combined

Zagonari

2019

Publications

View full text Add to dashboard Cite

In this study, I developed operational versions of Gini’s and Hirsch’s indexes that can be applied to characterize each researcher’s publication history (PH) as heterodox, orthodox, and interdisciplinary. In addition, the new indicators warn against anomalies that potentially arise from tactical or opportunistic citation and publication behaviors by authors and editors, and can be calculated from readily available information. I split the original Hirsch index into nested indexes to isolate networking activity, as well as to distinguish scientific production (number of articles) from scientific productivity (rate of production), and used nested Gini indexes to identify intentional and successful intertopical and interdisciplinary research. I applied the most popular standardizations (i.e., per author and per year), and used simple methodologies (i.e., least-squares linear and cubic fitting, whole-career vs. subperiods, two-dimensional graphs). I provide three representative numerical examples based on an orthodox multidisciplinary PH, a heterodox PH from the social sciences, and an orthodox unidisciplinary PH from the physical sciences. Two additional numerical examples based on PHs from the life and health sciences show that the suggested PH characterization can be applied to different disciplines where different publication and citation practices prevail. Software is provided to help readers explore the use of these indicators.

show abstract

“…Whereas sometimes impact is considered as a reliable proxy of quality (e.g. in the Italian VQR it is done for many scientific fields, especially in hard and life sciences), this identification is problematic, as it provides strong disincentives to work on mainstream problems and within heterodox schools of thought (as documented for instance by Castellani, Pontecorvo, and Valente 2016). To understand why this happens, consider the following scenario: two papers of comparable quality, P1 and P2, provide a relevant insight over a same issue.…”

Section: Bibliometricsmentioning

confidence: 99%

Evaluation of Research(ers) and its Threat to Epistemic Pluralisms

Viola

2018

EuJAP

View full text Add to dashboard Cite

While some form of evaluation!has always been employed in science (e.g. peer review, hiring), formal systems of evaluation of research and researchers have recently come to play a more prominent role in many countries because of the adoption of new models of governance. According to such models, the quality of the output of both researchers and their institutions is measured, and issues such as eligibility for tenure or the allocation of public funding to research institutions crucially depends on the outcomes of such measures. However, concerns have been raised over the risk that such evaluation may be threatening epistemic pluralism by penalizing the existent heterodox schools of thought and discouraging the pursuit of new ones. It has been proposed that this may happen because of epistemic bias favouring mainstream research programmes. In this paper, I claim that (1) epistemic pluralism is desirable and should be preserved; (2) formal evaluation exercises may threaten epistemic pluralism because they may be affected by some form of epistemic bias; therefore, (3) to preserve epistemic pluralism, we need some strategy to actively dampen epistemic bias.

show abstract

Epistemic Consequences of Bibliometrics-based Evaluation: Insights from the Scientific Community

Cited by 10 publications

References 56 publications

Role taxonomy of green and sustainable science and technology journals: exportation, importation, specialization and interdisciplinarity

Role taxonomy of green and sustainable science and technology journals: exportation, importation, specialization and interdisciplinarity

Scientific Production and Productivity for Characterizing an Author’s Publication History: Simple and Nested Gini’s and Hirsch’s Indexes Combined

Evaluation of Research(ers) and its Threat to Epistemic Pluralisms

Contact Info

Product

Resources

About