Validity of citation criteria for assessing the influence of scientific publications: New evidence with peer assessment

Abstract

This article reviews the principal correlational studies employing citation counts as criterion measures for assessing the impact of scientific scholarship. The rationale and limitations of such measures and studies are discussed. New evidence on the validity of citation criteria is presented based on a sample of 870 cancer research papers, divided into three groups ("first-order" papers, abstracted in the Year Book of Cancer; "secondorder" papers, listed but not abstracted in the yearbook; and "average-order" papers, a representative cross section of research papers unlisted in the yearbook). Results consistently show that highly rated papers are more highly cited over the ensuing five years after publication, or when controls are introduced for selfcitations, for the influence of listing in the yearbook, and for language and country of authorship. The implicatlons of results are discussed.

Information science and scientists: JASIS, 1972-1990

“…The legitimacy of citation analysis for evaluation has been strengthened by research documenting a high correlation of citation counts with other measures of quality (Anderson, Narin, & McAllister, 1978;Koenig, 1983;Lawani & Bayer, 1983;Narin, Piniski, & Gee, 1976).…”

An empirical investigation of the influence of collaboration in Finance on article impact

“…Citation rates have also been shown to be associated with other indicators of quality such as number of scientific awards received and choices of expert panels (Cole and Cole 1973;Lawani 1977;Lawani and Bayer 1983).…”

Untitled

“…Even the founder of Thomson Reuters' Institute for Scientific Information (ISI), Eugene Garfield, holds that reading each article for its quality is actually essential for a reliable evaluation system despite its inevitably subjective nature (Garfield, 1994b). While citation rates can act as proxies for the impact of a piece of scholarship, (Lawani and Bayer, 1983), citation indices themselves are increasingly viewed as less than objective. Many underlying assumptions are deemed to no longer hold true in today's globalized academia, in particular that the influence of ISI-indexed journals is overstated and that the very word 'global' conceals the highly-localised master journal list (Cruz, 2007).…”