Targeted, actionable and fair: reviewer reports as feedback and its effect on ECR career choices

Abstract

Background: Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the ‘best’ decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants career decisions to persist (reapply for funding), or to switch (not to reapply, or else leave academia).Methods: This study combined data from a survey of all applicants to ECR funding calls of The Wellcome Trust 2009-2019; with interviews with unsuccessful applicants (n=19), and manual coding of reviewer comments received by applicants (n=81).Results: Survey results show a high proportion of resubmissions, with 50% of unsuccessful applicants at T0 reapplying at T1, and with 35% of these achieving success with this reapplication. Here, peer-review-comments-as-feedback sends signals to applicants to encourage them to persist (continue) or switch (not continue) even when the initial application has failed. Feedback associated by unsuccessful applicants as motivating their decision to resubmit had 3 characteristics: actionable; targeted; and fair.Conclusions: The results lead to identification of standards of feedback for funding agencies and peer-reviewers to promote when providing reviewer feedback to applicants as part of their peer review process. The provision of quality reviewer-reports-as-feedback to applicants, ensures that peer review acts as a participatory research governance tool focused on supporting the development of individuals and their future research plans.