The Role of Scientometric Thresholds for the Evaluation of Grant Applications

Abstract

The present study focuses on data from the Russian Science Foundation (RSF). The authors analyze the effect of using quantitative indicators in grant allocation by using the natural experiment with the increasing publication threshold for principal investigators between two waves of grant selections in 2014 and 2017. The authors selected the relatively new RSF as our case study due to its policy to establish a publication threshold for grants’ principal investigators. The policy change provides the authors with the opportunity to study whether reliance on bibliometric indicators brings better results in the project evaluation process. This analysis included two groups of researchers: 1) physicists and 2) social sciences and humanities scholars. Scopus was sourced to collect bibliographic data, while the foundation’s website was used to check data on the funded projects. The following questions are explored in detail: whether the policy affected the distribution of funds to researchers with a better publication record, the strategies of increasing publications by individual researchers, and the differences, if any, in policy effects between disciplines. The authors found that the selection among physicists in the first wave was already effective as the grant recipients are prolific authors who publish many highly cited papers before 2014. In addition, the results indicated that the group of research leaders in physics did not significantly change between the two selected waves of competitions (from 2014 to 2017). Although social scientists demonstrated a relatively weak ability to publish internationally, the increase in scientometric expectations has improved the publication record regarding the quantity and quality of publications.