Turning the tables on citation analysis one more time: Principles for comparing sets of documents

Abstract

We submit newly developed citation impact indicators based not on arithmetic averages of citations but on percentile ranks. Citation distributions are-as a rule-highly skewed and should not be arithmetically averaged. With percentile ranks, the citation of each paper is rated in terms of its percentile in the citation distribution. The percentile ranks approach allows for the formulation of a more abstract indicator scheme that can be used to organize and/or schematize different impact indicators according to three degrees of freedom: the selection of the reference sets, the evaluation criteria, and the choice of whether or not to define the publication sets as independent. Bibliometric data of seven principal investigators (PIs) of the Academic Medical Center of the University of Amsterdam is used as an exemplary data set. We demonstrate that the proposed indicators [R(6), R(100), R(6,k), R(100,k)] are an improvement of averages-based indicators because one can account for the shape of the distributions of citations over papers.

Which are the "best" cities for psychology research worldwide?

“…The share of top-10% most highly cited publications in their subject category (publication year, document type) has become a de facto standard for the assessment of excellence at the institutional level (Bornmann, de Moya Anegón, & Leydesdorff, 2012;Bornmann, Mutz, Marx, Schier, & Daniel, 2011;Leydesdorff, Bornmann, Mutz, & Opthof, 2011;Tijssen & van Leeuwen, 2006;Tijssen, Visser, & van Leeuwen, 2002;Waltman et al, 2012). In this study we follow this classification and focus on the top-10% of papers published In a first step, all papers (n = 21,528) with the document type "article" published in 2007 and belonging to the subject categories "psychology," "psychology, applied," "psychology, biological," "psychology, clinical," "psychology, developmental," "psychology, educational," "psychology, experimental," "psychology, mathematical," "psychology, multidisciplinary," "psychology, psychoanalysis," and "psychology, social" were downloaded from the WoS (Social Science Citation Index).…”

Ranking authors using fractional counting of citations: An axiomatic approach

“…Nonparametric approaches based on percentiles have also been investigated (Bornmann, 2013, Bornmann, Leydesdorff, and Mutz, 2013, Leydesdorff and Bornmann, 2012, Leydesdorff, Bornmann, Mutz, and Opthof, 2011. All approaches relating to this first path require to be able to allocate papers to fields.…”

“…It is described in Opthof (2010a, p. 2367, col. 2) as "simple and elegant" and in Leydesdorff, Radicchi, Bornmann, Castellano, andde Nooy (2013a, p. 2300, col. 1) as a "radicalized" version of the other techniques.…”

“…There has been a number of empirical studies on fractional counting of citations. Although some of them give evidence in its favor (Leydesdorff and Shin, 2011, Leydesdorff, Zhou, and Born-mann, 2013b, Zhou and Leydesdorff, 2011, others lead to a less positive picture (Bornmann and Marx, 2015, Leydesdorff et al, 2013a, Radicchi and Castellano, 2012a, Waltman and van Eck, 2013a.…”

Robustness of personal rankings: the Handelsblatt example

“…Therefore, it seems appropriate to standardize citation frequency with respect to field and time (Bornmann and Marx 2013). In particular, Leydesdorff et al (2011) suggest using percentile ranks that rate each paper in terms of its percentile in the citation distribution. Besides non-scientific motivations for citing a paper, citation analyses are criticized for leaving room for discretionary decisions, e.g., how to deal with self-citations or what the appropriate length of the considered citation window is (Abramo et al 2011).…”