Using categorisations of citations when assessing the outcomes from health research

Abstract

This paper describes an attempt to explore how far a categorisation of citations could be used as part of an assessment of the outcomes from health research. A large-scale project to assess the outcomes from basic, or early clinical, research is being planned, but before proceeding with such a project it was thought important to test and refine the developing methods in a preliminary study.Here we describe the development, and initial application, of one element of the planned methods: an approach to categorising citations with the aim of tracing the impact made by a body of research through several generations of papers. The results from this study contribute to methodological development for the large-scale project by indicating that: only for a small minority of citing papers is the cited paper of considerable importance; the number of times a paper is cited can not be used to indicate the importance of that paper to the articles that cite it; and self-citations could play an important role in facilitating the eventual outcomes achieved from a body of research.

Functions of Uni- and Multi-citations: Implications for Weighted Citation Analysis

“…Although some later studies (e.g. Hanney et al, 2005) found no significant difference in terms of citation location for citation importance, many studies found that citations located in the Methodology, Results, Discussion, or Conclusion sections may play a more significant or meaningful role than those located in the introductory sections (Bonzi, 1982;Cano, 1989;Tang & Safer, 2008;Voos & Dagaev, 1976).…”

Evaluating Research in Cardiovascular Medicine

“…38 -41 Indeed, a citation content analysis of 623 diabetes cardiovascular studies revealed that fewer than 10% of the cited references were deemed to be of considerable or essential importance to the citing study by independent assessors. 42 However, our bibliometric analyses did identify a highly cited group of studies on focused clinical topics, and our purpose in creating the lists of the top 100 cited papers for each intervention was not to analyze relative positions within the top 100 articles but instead to create a reference point against which to benchmark the landmark studies. To provide context for the citation counts we provide in Table 1, it is noteworthy that only 0.5% of the publications indexed in ISI have been cited more than 200 times, and half are never cited at all.…”

“…[38][39][40][41] En effet, une analyse des références bibliographiques citées à l'appui de 623 études sur les conséquences cardiovasculaires du diabète a fait apparaître que moins de 10 % des références produites avaient été jugées par des examinateurs indépendants comme étant d'un intérêt majeur ou essentiel pour les publications dans lesquelles elles figuraient. 42 Bien que nous ayons effectué une recherche systématique dans la base de données en ligne Web of Science et que deux investigateurs aient examiné séparément les résultats afin de recenser et de classer les publications ayant donné lieu au plus grand nombre de citations dans le cadre des six interventions thérapeutiques qui ont le plus influé sur la mortalité cardiovasculaire au cours des trente dernières années, notre étude présente plusieurs limites. En premier lieu, le lecteur est fondé à objecter que d'autres interventions ont également joué un rôle important dans l'amélioration du pronostic cardiovasculaire ; tel est notamment le cas de la création des unités de soins intensifs coronaires, du traitement anticoagulant chez les patients atteints de fibrillation auriculaire ou porteurs de prothèses valvulaires, des stimulateurs cardiaques et défibrillateurs automatiques implantables ainsi que de la transplantation cardiaque.…”

Current impact of ceased journals: are they still alive?

“…Comparing the history and contextual importance of eleven highly cited articles in the field of molecular genetics, McCain and Turner (1989) developed a classification scheme based on the assumption that a citation in a method section of a research paper was more important than one in the introduction section. Hanney et al (2005) designed a complex model for identifying the importance of citations in a paper combining the number of citation occurrences and their locations in the text with the reasons behind each of them (develop, support, apply, refute, note/review only). They classified the importance of the cited papers as peripheral, limited, considerable, and essential.…”