Journal metrics as predictors of Research Excellence Framework 2021 results: Comparison of impact factor quartiles and Finnish expert-ratings

Abstract

This study compares citation-based and expert-based journal metrics as predictors of peer-assessed research quality based on 154,826 journal articles submitted to UK’s Research Excellence Framework (REF) 2021. The Finnish expert-based Julkaisufoorumi (JUFO) level ratings of journals determined by expert-panels per field produce scores that correlate more strongly with REF scores than those based on citation-based Journal Impact Factor (JIF) or Journal Citation Indicator (JCI) Quartiles. This holds true at aggregate levels of 34 Subject areas, 157 Higher Education Institutions (HEI), and 1,888 Units of Assessment (UoA). Especially non-field-normalised JIF-based scores correlate poorly with REF scores. All types of journal metrics are more aligned with expert-based REF scores at the highest aggregate level of HEIs and agree less at the lower aggregate level of UoAs and Subject areas.