The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology

Oppenheim, Charles

doi:10.1108/eum0000000007207

Cited by 117 publications

(108 citation statements)

References 22 publications

(23 reference statements)

Supporting

Mentioning

105

Contrasting

Unclassified

Order By: Relevance

“…The task of assessing the RAE subject areas ("units of assessment" in RAE terminology) is very expensive both for the government and the universities that make submissions. In response to this, a proposal has been made by Oppenheim that an exercise relying mainly, but not exclusively, upon citation counting could prove much cheaper (Holmes & Oppenheim, 2001;Oppenheim, 1995Oppenheim, , 1997. This suggestion has been controversial (Warner, 2000a(Warner, , 2000b, but there is agreement that a high citation count would seem to be at least a logical source of additional evidence for the importance of research publications.…”

Section: Academic Institutions and Departmentsmentioning

confidence: 99%

“…There have been successful attempts in the UK to correlate research ratings with traditional citation counts for four subject areas, including information science (Oppenheim, 1995(Oppenheim, , 1997. The definitive ratings for research in UK academic institutions are given by the five yearly government Research Assessment Exercise (RAE).…”

Section: Academic Institutions and Departmentsmentioning

confidence: 99%

See 1 more Smart Citation

Extracting macroscopic information from Web links

Thelwall

2001

J. Am. Soc. Inf. Sci.

154

173

View full text Add to dashboard Cite

Much has been written about the potential and pitfalls of macroscopic Web-based link analysis, yet there have been no studies that have provided clear statistical evidence that any of the proposed calculations can produce results over large areas of the Web that correlate with phenomena external to the Internet. This article attempts to provide such evidence through an evaluation of Ingwersen's (1998) proposed external Web Impact Factor (WIF) for the original use of the Web: the interlinking of academic research. In particular, it studies the case of the relationship between academic hyperlinks and research activity for universities in Britain, a country chosen for its variety of institutions and the existence of an official government rating exercise for research. After reviewing the numerous reasons why link counts may be unreliable, it demonstrates that four different WIFs do, in fact, correlate with the conventional academic research measures. The WIF delivering the greatest correlation with research rankings was the ratio of Web pages with links pointing at research-based pages to faculty numbers. The scarcity of links to electronic academic papers in the data set suggests that, in contrast to citation analysis, this WIF is measuring the reputations of universities and their scholars, rather than the quality of their publications.

show abstract

Section: Academic Institutions and Departmentsmentioning

confidence: 99%

Section: Academic Institutions and Departmentsmentioning

confidence: 99%

Extracting macroscopic information from Web links

Thelwall

2001

J. Am. Soc. Inf. Sci.

154

173

View full text Add to dashboard Cite

show abstract

“…In particular the results are less reliable than citation counting based exercises, which have been proposed as the basis for an alternative to the RAE in Britain (Oppenheim, 1997;Warner, 2000;Holmes & Oppenheim, 2001). The analysis of individual results was, however, able to identify sources of variation, but it is possible that had it been pursued systematically, improved results could have been obtained.…”

Section: Discussionmentioning

confidence: 95%

A comparison of sources of links for academic Web impact factor calculations

Thelwall

2002

Journal of Documentation

View full text Add to dashboard Cite

There has been much recent interest in extracting information from collections of web links. One tool that has been used is Ingwersen's Web Impact Factor. It has been demonstrated that several versions of this metric can produce results that correlate with research ratings of British universities showing that, despite being a measure of a purely Internet phenomenon, the results are susceptible to a wider interpretation. This paper addresses the question of which is the best possible domain to count backlinks from, if research is the focus of interest. WIFs for British universities calculated from several different source domains are compared, primarily the .edu, .ac.uk and .uk domains, and the entire web. The results show that all four areas produce WIFs that correlate strongly with research ratings, but that none produce incontestably superior figures. It was also found that the WIF was less able to differentiate in more homogenous subsets of universities, although positive results are still possible.

show abstract

“…Moreover, there is an increasing body of evidence to support the view that bibliometric approaches can yield results closely mirroring the judgments of subject experts. Specifically, significant correlations have been observed between expert judgments and bibliometric data in comparisons carried out in Italy (Abramo et al, 2009;Reale et al, 2007), the Netherlands (Rinia et al, 1998;van Raan, 2006) and the UK (Norris and Oppenheim, 2003;Oppenheim, 1997;Seng and Willett, 1995). That said, bibliometrics has its own limitations, and these would have to be borne in mind if a decision was taken at some point in the future to base research evaluation solely on bibliometric data, without recourse to expert judgments.…”

Section: Introductionmentioning

confidence: 99%

Computer science research in Malaysia: a bibliometric analysis

Bakri

Willett

2011

Aslib Proceedings

View full text Add to dashboard Cite

Purpose -To analyse the publications of, and the citations to, the current staff of 19 departments of computer science in Malaysian universities, and to compare this bibliometric data with expert peer reviews of Malaysian research performance. Method -Author and citation searches of the Scopus and Web of Science databases.Findings -Both publication and citation rates are low, although this is at least in part due to some Malaysian universities having only a teaching function. More of the departments' publications were identified in Scopus than in Web of Science, but both databases were needed for comprehensive coverage. Statistically significant relationships were observed between the departments' publication and citation counts and the rankings of the departments' parent universities in two evaluations of the research performance of Malaysian universities Originality -This is the first comparison of bibliometric and peer-review data for Malaysia, and, more generally, for a country with a newly developed higher education system. Keywords - IntroductionGovernments worldwide are looking for ways in which they can evaluate the quality of the research that is carried out in their countries' universities. Informal evaluations have been carried out for many years, but the increasing costs of higher education provision have resulted in the development of more formal evaluation mechanisms. These mechanisms are designed to ensure that government funding will be channelled to those institutions and research groups that have demonstrated their ability to carry out high-quality research in a cost-effective manner. assessments that focus on specific disciplines or groups of disciplines.Research quality has traditionally been assessed by means of expert review (in a manner analogous to the procedures used for refereeing journal articles and grant applications), not least because this approach is well established and generally enjoys the support of the academic community. It is, however, very costly in terms of the time of the subject experts, and this has spurred interest in the use of bibliometric indicators as a surrogate for peer review. These indicators can consider both the quantity of research (as reflected in the numbers of research publications produced by a university, department or whatever) and the quality of research (as reflected in the numbers of citations to those publications) and are typically far cheaper to use since they exploit existing bibliometric databases and do not require costly human judgments. Moreover, there is an increasing body of evidence to support the view that bibliometric approaches can yield results closely mirroring the judgments of subject experts. Specifically, significant correlations have been observed between expert judgments and bibliometric data in comparisons carried out in Italy (Abramo et al

show abstract

The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology

Cited by 117 publications

References 22 publications

Extracting macroscopic information from Web links

Extracting macroscopic information from Web links

A comparison of sources of links for academic Web impact factor calculations

Computer science research in Malaysia: a bibliometric analysis

Contact Info

Product

Resources

About