The acceptability of using a lottery to allocate research funding: a survey of applicants

Abstract

Background: The Health Research Council of New Zealand is the first major government funding agency to use a lottery to allocate research funding for their Explorer Grant scheme. This is a somewhat controversial approach because, despite the documented problems of peer review, many researchers believe that funding should be allocated solely using peer review, and peer review is used almost ubiquitously by funding agencies around the world. Given the rarity of alternative funding schemes, there is interest in hearing from the first cohort of researchers to ever experience a lottery. Additionally, the Health Research Council of New Zealand wanted to hear from applicants about the acceptability of the randomisation process and anonymity of applicants. Methods: This paper presents the results of a survey of Health Research Council applicants from 2013 to 2019. The survey asked about the acceptability of using a lottery and if the lottery meant researchers took a different approach to their application. Results: The overall response rate was 39% (126 of 325 invites), with 30% (76 of 251) from applicants in the years 2013 to 2018, and 68% (50 of 74) for those in the year 2019 who were not aware of the funding result. There was agreement that randomisation is an acceptable method for allocating Explorer Grant funds with 63% (n = 79) in favour and 25% (n = 32) against. There was less support for allocating funds randomly for other grant types with only 40% (n = 50) in favour and 37% (n = 46) against. Support for a lottery was higher amongst those that had won funding. Multiple respondents stated that they supported a lottery when ineligible applications had been excluded and outstanding applications funded, so that the remaining applications were truly equal. Most applicants reported that the lottery did not change the time they spent preparing their application. Conclusions: The Health Research Council's experience through the Explorer Grant scheme supports further uptake of a modified lottery.

Face-to-face panel meetings versus remote evaluation of fellowship applications: simulation study at the Swiss National Science Foundation

“…Of note, panels applied the random selection only in a few cases, in 8 (6.0%) of 134 applications. In the context of the Explorer Grant scheme of the HRC-NZ, Liu et al 28 recently reported that most applicants agreed with the use of a random selection. In this study, the SNSF received a few unsolicited questions about the procedure but otherwise no negative or positive reactions to the use of random selection were received from applicants.…”

Administrative discretion in scientific funding: Evidence from a prestigious postdoctoral training program✰

“…Short proposals are screened for eligibility and then funded at random until the budget is exhausted. A recent study reported that those who were funded by the lottery supported this system, however, it did not evaluate the impact of receiving funding on scientific output (Liu et al 2020).…”

How to best evaluate applications for junior fellowships? Remote evaluation and face-to-face panel meetings compared

“…In the context of the Explorer Grant scheme of the HRC-NZ, Liu et. al [28] recently reported that most applicants agreed with the use of a random selection. In this study, no negative or positive reactions to the use of random selection were received from applicants.…”