The Virtues of Virtual Panels

What makes an effective grants peer reviewer? An exploratory study of the necessary skills

“…It is unclear whether differences exist in review quality across formats and which format is "better" [16,29], but Pier et al [14,16] did note slightly greater efficiency of reviewers' time in face-to-face meetings compared to virtual meetings. Venkatraman [15] found a greater amount of, and more valuable, reviewer participation in face-to-face settings compared to virtual settings. Carpenter et al [1] noted small but statistically significant differences between settings (face-to-face vs. teleconference, no visual component) in terms of its effect on discussion and posit "teleconference reviewers possibly being slightly less engaged than those participating onsite."…”

“…Online panels save money and allow a greater number and variety of reviewers, including international scientists, to participate. Within virtual panels the work of evaluation is accomplished, "applications are read, and decisions are taken efficiently" [15]. The Canadian Institutes of Health Research [50] shifted from in-person to online reviews in 2014 to, among other things, "gain cost-effective access to a broader base of expertise (including international experts)."…”

“…Venkatraman [15] found that participation and the ability to recruit high-level reviewers increases with technological options. Online reviews enable the inclusion of a greater variety of researchers.…”

“…Webster [52] writes, "when you are together in a room, you are much more committed to the process than when it's online." Networking and collaborations can be missed in virtual panels, confidentiality is challenged, technological difficulties exist, and major disagreements are harder to resolve [15]. Similarly, virtual discussions may remove productive debate, support biases towards established researchers, and disallow creative discussion and collective visioning for a field [50].…”

“…Some authors have pointed to the importance of the review process [12] and the people serving as panelists [13],-either the need to identify and understand them nor their significance to the success of peer review panels. When panel format has been examined, it has been from the perspective of determining if using technology produces better outcomes [14,15] in decision-making [16] or merit scores, score distribution, or reviewer demographics [17,18]. Answers have been inconclusive and resulted in numerous calls for further research on the effects of panel format, yet these suggestions of additional investigation rarely reference peer review skills.…”