Overburdening of peer reviewers. A multi-disciplinary and multi-stakeholder perspective on causes, effects and potential policy implications

Abstract

Peer review of manuscripts is labour-intensive and time-consuming. Individual reviewers often feel themselves overburdened with the amount of reviewing they are requested to do. Aiming to explore how stakeholder groups perceive reviewing burden and what they believe to be the causes of a potential overburdening of reviewers, we conducted focus groups with early-, mid-, and senior career scholars, editors, and publishers. By means of a thematic analysis, we aimed to identify the causes of overburdening of reviewers. First, we show that, across disciplines and roles, stakeholders believed that the reviewing workload has become so enormous that the academic community is no longer able to supply the reviewing resources necessary to address its demand for peer review. Second, the reviewing workload is distributed unequally across the academic community, thereby overwhelming small groups of scholars. Third, stakeholders believed the overburdening of reviewers to be caused by (i) an increase in manuscript submissions; (ii) insufficient editorial triage; (iii) a lack of reviewing instructions; (iv) difficulties in recruiting reviewers; (v) inefficiencies in manuscript handling and (vi) a lack of institutionalisation of peer review. These themes were assumed to mutually reinforce each other and to relate to an inadequate incentive structure in academia that favours publications over peer review. In order to alleviate reviewing burden, a holistic approach is required that addresses both the increased demand for and the insufficient supply of reviewing resources.

Diverse Reviewer Suggestion for Extending Conference Program Committees

“…A value κ describes the number of fitting inserted reviewers per iteration (l. 10). If a feasible assignment (l. 7) has been found for a reviewer set, we randomly exclude reviewers from reviewing submissions in order to find the most diverse assignment (l. [15][16][17][18][19][20].…”

“…Consequences of the mismatch might result in manuscripts tackling topics far from the PC's interests being less favourably reviewed [16] and a general overburdening of reviewers. This, in turn, might lead to innovative and complex submissions being rejected solely due to low-quality reviews [1] or failure to find errors in submissions [20].…”

“…This can especially help to cover new or emerging research topics [6] and to ensure that under-represented groups can gain exposure and reviewing experience [19]. Identification of appropriate candidates is challenging as PC members should be diverse in localities, seniority [20,16], research topics and gender [16]. Furthermore, suggested candidates should be explainable, in order to aid the conference chairs in effective and efficient decision making.…”