Multi-Round compared to Real-Time Delphi for consensus in core outcome set (COS) development: a randomised trial

Abstract

Background The Delphi method is used in a wide variety of settings as a method of building consensus on important issues. Traditionally, the Delphi method uses multiple rounds of a survey to allow for feedback of other participants’ survey responses in between rounds. By informing participants about how others answer a question or prioritise specific topics, it allows for diverse opinions to inform the consensus process. For this reason, the Delphi method is popular as a consensus building approach in developing core outcome sets (COS), i.e. the minimum agreed set of standardised outcomes that should be measured and reported in studies on a specific health condition. In a COS setting, participants prioritise the importance of outcomes for inclusion in a COS. This usually involves participating in multiple rounds of a survey that can span several weeks or months. Challenges with participant retention have been highlighted in previous COS. We will compare a three-round with a Real-Time Delphi approach on prioritised outcomes. This trial is embedded within the COHESION study which is developing a COS for interventions treating neonatal encephalopathy. Methods One hundred and eighty stakeholders (parents/caregivers of infants diagnosed and treated with neonatal encephalopathy, healthcare providers and researchers) will be randomised using stratified randomisation to take part in either the Multi-Round or Real-Time Delphi. Stakeholders will rate the importance of the same set of outcomes in both arms. We will compare the prioritised outcomes at the end of both surveys as well as other parameters such as feedback, initial condition and iteration effects. Discussion This trial will provide evidence to inform decisions on the use of Multi-Round compared to Real-Time Delphi survey methods. Trial registration NCT04471103. Registered on 14 July 2020.

Core outcomes in neonatal encephalopathy: a qualitative study with parents

“…Although many outcomes were identified as important by parents from both LMiCs and HiCs, some were unique to either LMiC or HiC. While, the reasons for these differences in outcomes is beyond the scope of this work, all outcomes will be included in the next phase of the COS development 6 33…”

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

“…RTD has been used in healthcare studies that aim to develop a medication adherence technologies repository [ 50 ] and determine the negative impacts of substance use disorders among people with HIV in the United States [ 51 ]. Several researchers have incorporated an RTD approach into the COS development processes [ 52 , 53 ].…”

Developing a medication adherence technologies repository: proposed structure and protocol for an online real-time Delphi study

“…The RT-Delphi approach was developed by Gordon and Pease to improve efficiency of the process and shorten the time of performance 39. Since then, several online tools have been developed to facilitate the RT-Delphi design40 and literature describing the use of RT-Delphi and comparison with the traditional multiround Delphi approach is growing 23 41–44. In contrast to the traditional Delphi, the real-time approach is round-less and offers a constant iteration by providing immediate (real-time) individual and aggregated feedback.…”

“…Based on new information participants can rethink and modify their answers, which could lead to reconciliation of opinions and eventually to consensus. Participants are encouraged to revisit and engage in the survey several times during the study period 39 40 42 44. In comparison with the traditional approach, the real-time approach encompasses all key Delphi features43 and is similar from all key perspectives 23 41 43 44.…”