Joint international consensus statement on crowdsourcing challenge contests in health and medicine: results of a modified Delphi process

Abstract

ObjectivesTo develop a consensus statement to provide advice on designing, implementing and evaluating crowdsourcing challenge contests in public health and medical contexts.DesignModified Delphi using three rounds of survey questionnaires and one consensus workshop.SettingUganda for face-to-face consensus activities, global for online survey questionnaires.ParticipantsA multidisciplinary expert panel was convened at a consensus-development conference in Uganda and included 21 researchers with experience leading challenge contests, five public health sector workers, and nine Ugandan end users. An online survey was sent to 140 corresponding authors of previously published articles that had used crowdsourcing methods.ResultsA subgroup of expert panel members developed the initial statement and survey. We received responses from 120 (85.7%) survey participants, which were presented at an in-person workshop of all 21 panel members. Panelists discussed each of the sections, revised the statement, and participated in a second round of the survey questionnaire. Based on this second survey round, we held detailed discussions of each subsection with workshop participants and further revised the consensus statement. We then conducted the third round of the questionnaire among the 21 expert panelists and used the results to finalize the statement. This iterative process resulted in 23 final statement items, all with greater than 80% consensus. Statement items are organised into the seven stages of a challenge contest, including the following: considering the appropriateness, organising a community steering committee, promoting the contest, assessing contributions, recognising contributors, sharing ideas and evaluating the contest (COPARSE).ConclusionsThere is high agreement among crowdsourcing experts and stakeholders on the design and implementation of crowdsourcing challenge contests. The COPARSE consensus statement can be used to organise crowdsourcing challenge contests, improve the rigour and reproducibility of crowdsourcing research and enable large-scale collaboration.

Social innovation in health, community engagement, financing and outcomes: qualitative analysis from the social innovation in health initiative

“…More details about crowdsourcing open calls can be found in the TDR/SESH/SIHI practical guide 10. The network has consensus guidelines on implementing open calls 11. The first crowdsourcing open call took place in 2015, with subsequent regional open calls during 2017–2018.…”

Audio innovation and songs to spur change in global health: evidence from a national crowdsourcing open call for youth social innovation in Malaysia

“…The open call accepted submissions from 23 March 2021 to 7 May 2021 (see online supplemental table 1). The entire open call process was guided by the practical guide on crowdsourcing in health and health research that was developed by WHO/TDR (the Special Programme for Research and Training in Tropical Diseases) and SESH in collaboration with SIHI26 and the SIHI consensus statement on open calls 27. In the open call, Malaysian youth aged 18–30 years old were invited to submit entries in English or Bahasa Malaysia.…”

Research Mentorship in Low and Middle-Income Countries: A Global Qualitative Evidence Synthesis of Data from a Crowdsourcing Open Call and Scoping Review

“…The open call was implemented using stages outlined in the in the TDR/SESH/SIHI practical guide on crowdsourcing. 6,7 The open call was launched on 21 st (4) recognized finalist participants; (5) refined ideas and implemented selected ideas (Table 1).…”