Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization

Bornmann, Lutz; Mutz, Rüdiger

doi:10.1016/j.joi.2010.10.009

Cited by 111 publications

(73 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A reply to Opthof and Leydesdorff was given by CWTS (Van Raan et al 2010). Other contributions to the discussion were made by Bornmann (2010), Bornmann and Mutz (2011), Gingras and Larivière (2011), Leydesdorff and Opthof (2010, 2011), Moed (2010), and Spaan (2010). Indicators that rely on the alternative normalization mechanism are being used by various institutes, among which Karolinska Institute in Sweden (Rehn and Kronman 2008), Science-Metrix in the US and Canada (e.g., Campbell et al 2008, p. 12), the SCImago research group in Spain (SCImago Research Group 2009), and Wageningen University in the Netherlands (Van Veller et al 2009).…”

Section: Introductionmentioning

confidence: 99%

Towards a new crown indicator: an empirical analysis

et al. 2011

View full text Add to dashboard Cite

We present an empirical comparison between two normalization mechanisms for citation-based indicators of research performance. These mechanisms aim to normalize citation counts for the field and the year in which a publication was published. One mechanism is applied in the current so-called crown indicator of our institute. The other mechanism is applied in the new crown indicator that our institute is currently exploring. We find that at high aggregation levels, such as at the level of large research institutions or at the level of countries, the differences between the two mechanisms are very small. At lower aggregation levels, such as at the level of research groups or at the level of journals, the differences between the two mechanisms are somewhat larger. We pay special attention to the way in which recent publications are handled. These publications typically have very low citation counts and should therefore be handled with special care.

show abstract

Section: Introductionmentioning

confidence: 99%

Towards a new crown indicator: an empirical analysis

et al. 2011

View full text Add to dashboard Cite

show abstract

“…Two decisions have to be made: (1) which measure to use as the expected citation rate (e.g., the arithmetic mean), and (2) how to delineate the set of documents that will serve as the reference set. Since citation distributions are heavily skewed, one can build a strong case against the mean and in favor of non-parametric statistics (Bornmann and Mutz 2011; Leydesdorff et al 2011; Seglen 1992). Non-parametric statistics, however, are often less intuitively accessible for managers and evaluees.…”

Section: Ambivalences Around the Datamentioning

confidence: 99%

Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report

2016

Self Cite

View full text Add to dashboard Cite

Bibliometric indicators such as journal impact factors, h-indices, and total citation counts are algorithmic artifacts that can be used in research evaluation and management. These artifacts have no meaning by themselves, but receive their meaning from attributions in institutional practices. We distinguish four main stakeholders in these practices: (1) producers of bibliometric data and indicators; (2) bibliometricians who develop and test indicators; (3) research managers who apply the indicators; and (4) the scientists being evaluated with potentially competing career interests. These different positions may lead to different and sometimes conflicting perspectives on the meaning and value of the indicators. The indicators can thus be considered as boundary objects which are socially constructed in translations among these perspectives. This paper proposes an analytical clarification by listing an informed set of (sometimes unsolved) problems in bibliometrics which can also shed light on the tension between simple but invalid indicators that are widely used (e.g., the h-index) and more sophisticated indicators that are not used or cannot be used in evaluation practices because they are not transparent for users, cannot be calculated, or are difficult to interpret.

show abstract

“…However, recently this approach to normalisation has been criticised ( Leydesdorff & Opthoft, 2011;Lundberg, 2007;Opthof & Leydesdorff, 2010) and an alternative has been used in several cases (Cambell, Archambaulte, & Cote, 2008;Rehn & Kronman, 2008;Van Veller, Gerritsma, Van der Togt, Leon, & Van Zeist, 2009). This has generated considerable debate in the literature (Bornmann, 2010;Bornmann & Mutz, 2011;Moed, 2010;Waltman, van Eck, van Leeuwen, Visser, & van Raan, 2010, 2011. The alternative method calculates the expected number of citations for a field in the same way but then, instead of summing the actual citations and the expected citations and then dividing the two, it performs the division first for each paper.…”

Section: The "Crown Indicator"mentioning

confidence: 99%

Evaluating a department’s research: Testing the Leiden methodology in business and management

Mingers¹,

Lipitakis²

2013

Information Processing & Management

View full text Add to dashboard Cite

The Leiden methodology (LM), also sometimes called the "crown indicator", is a quantitative method for evaluating the research quality of a research group or academic department based on the citations received by the group in comparison to averages for the field. There have been a number of applications but these have mainly been in the hard sciences where the data on citations, provided by the ISI Web of Science (WoS), is more reliable. In the social sciences, including business and management, many journals and books are not included within WoS and so the LM has not been tested here. In this research study the LM has been applied on a dataset of over 3,000 research publications from three UK business schools. The results show that the LM does indeed discriminate between the schools, and has a degree of concordance with other forms of evaluation, but that there are significant limitations and problems within this discipline.

show abstract

Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization

Cited by 111 publications

References 8 publications

Towards a new crown indicator: an empirical analysis

Towards a new crown indicator: an empirical analysis

Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report

Evaluating a department’s research: Testing the Leiden methodology in business and management

Contact Info

Product

Resources

About