The first Italian research assessment exercise: A bibliometric perspective

Abstract

a b s t r a c tIn December 2003, seventeen years after the first UK research assessment exercise, Italy started up its first-ever national research evaluation, with the aim to evaluate, using the peer review method, the excellence of the national research production. The evaluation involved 20 disciplinary areas, 102 research structures, 18,500 research products and 6661 peer reviewers (1465 from abroad); it had a direct cost of 3.55 millions Euros and a time length spanning over 18 months. The introduction of ratings based on ex post quality of output and not on ex ante respect for parameters and compliance is an important leap forward of the national research evaluation system toward meritocracy. From the bibliometric perspective, the national assessment offered the unprecedented opportunity to perform a large-scale comparison of peer review and bibliometric indicators for an important share of the Italian research production. The present investigation takes full advantage of this opportunity to test whether peer review judgements and (article and journal) bibliometric indicators are independent variables and, in the negative case, to measure the sign and strength of the association. Outcomes allow us to advocate the use of bibliometric evaluation, suitably integrated with expert review, for the forthcoming national assessment exercises, with the goal of shifting from the assessment of research excellence to the evaluation of average research performance without significant increase of expenses.

Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)

“…Although some studies suggest that indicator-based systems are cheaper than peer review assessment (Franceschet and Costantini, 2011), we found no published material detailing the costs of a metrics-based system. To fill this gap, we interviewed managers and experts in Norway and Sweden where metrics-based systems are used to allocate part of their research budgets.…”

A avaliação da produção científica nas subáreas da Saúde Coletiva: limites do atual modelo e contribuições para o debate

“…Essa revisão é necessária, sobretudo, para que se evite a cristalização de parâmetros e critérios e para que se discutam criticamente as consequências que produzem, evitando distorções 6,7 . A avaliação tem um papel indutor importante no desenvolvimento da área, pois os critérios utilizados orientam os caminhos para a organização dos programas e apontam tendências.…”

Bibliometric Analysis of the Inkom Journal (Analisis Bibliometrik Jurnal Inkom)

“…Development of science is frequently related to scientific productivity, the ability to produce a scientific publication (Franceschet and Costantini, 2011). The publication can be measured through bibliometric indicators (Piro, F.N., 2011).…”