What do citation counts measure? A review of studies on citing behavior

Abstract

PurposeThe purpose of this paper is to present a narrative review of studies on the citing behavior of scientists, covering mainly research published in the last 15 years. Based on the results of these studies, the paper seeks to answer the question of the extent to which scientists are motivated to cite a publication not only to acknowledge intellectual and cognitive influences of scientific peers, but also for other, possibly non‐scientific, reasons.Design/methodology/approachThe review covers research published from the early 1960s up to mid‐2005 (approximately 30 studies on citing behavior‐reporting results in about 40 publications).FindingsThe general tendency of the results of the empirical studies makes it clear that citing behavior is not motivated solely by the wish to acknowledge intellectual and cognitive influences of colleague scientists, since the individual studies reveal also other, in part non‐scientific, factors that play a part in the decision to cite. However, the results of the studies must also be deemed scarcely reliable: the studies vary widely in design, and their results can hardly be replicated. Many of the studies have methodological weaknesses. Furthermore, there is evidence that the different motivations of citers are “not so different or ‘randomly given’ to such an extent that the phenomenon of citation would lose its role as a reliable measure of impact”.Originality/valueGiven the increasing importance of evaluative bibliometrics in the world of scholarship, the question “What do citation counts measure?” is a particularly relevant and topical issue.

Las prácticas de citación como interpretantes semióticos de acreditación de saberes locales en astronomía: México 1952-1972

“…Este hecho ha estado acompañado de distintas interpretaciones sobre las funciones de las referencias en el texto científico. La literatura en bibliometría/cienciometría da cuenta de este debate (Cronin, 1981;Cozzens, 1989;Bornmann & Daniel, 2008).…”

What factors affect the visibility of Argentinean publications in humanities and social sciences in Scopus? Some evidence beyond the geographic realm of research

“…This means that the measures provided by bibliometrics are not the only indicators that might account for the results and impact of research. It follows that any measure used to gauge impact will also reflect social factors beyond the conventions or patterns of behavior of scientific output, which may be linked with the prestige of the author or of the institution where he/she works (Bornmann & Daniel, 2008), among other factors. Still, bibliometrics do reflect the cognitive value and scientific influence of the knowledge generated, and therefore are valid indicators of potential utility in benefit of the scientific, technological, economic and social development of the countries.…”

A suggested method for the measurement of world-leading research (illustrated with data on economics)

“…I include self-citations because there is a case for leaving them in and they make only a trivial difference in the case of highly-cited papers such as those covered here; I do not weight by the source of the citing journal; doing so would in my judgment be against the spirit of free competition among intellectual ideas. Nevertheless, I remain conscious of the difficulties and sociological influences pointed out by BORNMANN and DANIEL [2008].…”