Participating in core outcome set development via Delphi surveys: qualitative interviews provide pointers to inform guidance

Biggane, Alice M; Williamson, Paula; Ravaud, Philippe; Young, Bridget

doi:10.1136/bmjopen-2019-032338

Cited by 21 publications

(20 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our review indicates that the 1–9 scoring system is the most commonly used system in COS studies that include patients; however, this scoring system is used in the DelphiManager software, and the large number of electronically delivered studies that reported using this software may, therefore, have influenced this finding. Biggane et al 7 interviewed patients retrospectively about their experience of participating in a Delphi survey, noting that while there are statistical considerations influencing the choice of scoring scales, patients can have differing views on the scales used. While some patients in their study preferred the 1–9 scoring scale, others struggled to use it, indicating the need for additional support and guidance.…”

Section: Discussionmentioning

confidence: 99%

“…There are several challenges in including patient participants in COS, and indeed there are indications some COS developers ‘problematise’ patient participation, 6 highlighting, for example, the tendency for patients to rate many outcomes highly. Biggane et al 7 found that patients without prior experience of Delphi surveys expressed difficulty understanding both the purpose of the COS and particular aspects of the surveys. Young and Bagley 8 called for further research exploring how patient input is currently being sought in COS studies and to understand more about the challenges of including and engaging patients in COS development.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Patient participation in Delphi surveys to develop core outcome sets: systematic review

2021

Self Cite

View full text Add to dashboard Cite

ObjectivesTo describe the design and conduct of core outcome set (COS) studies that have included patients as participants, exploring how study characteristics might impact their response rates.DesignSystematic review of COS studies published between 2015 and 2019 that included more than one patient, carer or representative as participants (hereafter referred to as patients for brevity) in scoring outcomes in a Delphi.ResultsThere were variations in the design and conduct of COS studies that included patients in the Delphi process, including differing: scoring and feedback systems, approaches to recruiting patients, length of time between rounds, use of reminders, incentives, patient and public involvement, and piloting. Minimal reporting of participant characteristics and a lack of translation of Delphi surveys into local languages were found. Additionally, there were indications that studies that recruited patients through treatment centres had higher round two response rates than studies recruiting through patient organisations.ConclusionsVariability was striking in how COS Delphi surveys were designed and conducted to include patient participants and other stakeholders. Future research is needed to explore what motivates patients to take part in COS studies and what factors influence COS developer recruitment strategies. Improved reporting would increase knowledge of how methods affect patient participation in COS Delphi studies.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Patient participation in Delphi surveys to develop core outcome sets: systematic review

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…A modified Delphi technique was used to develop a minimal standard outcome set. The Delphi technique is an iterative multi stage process to actively transform opinion into group consensus [ 10 , 11 ].…”

Section: Methodsmentioning

confidence: 99%

A standard set of outcome measures for the comprehensive assessment of osteogenesis imperfecta

Nijhuis

Franken

Ayers

et al. 2021

Orphanet J Rare Dis

View full text Add to dashboard Cite

Background Osteogenesis Imperfecta (OI) is a genetic disorder also known as ‘brittle bone disease’. The clinical manifestation of OI shows a wide variation. Therefore, care for patients with OI requires an interdisciplinary approach. The effectiveness of particular interventions and treatment protocols of interdisciplinary teams is not clear due to a non-standardized and wide variation of patient outcomes thus making the comparison of outcome measures available in the literature difficult. It is only by agreeing on a common, standard set of outcome measures for the comprehensive appraisal of OI that comparisons across interdisciplinary treatment centers for OI will be possible in the future. Methods The Key4OI international interdisciplinary working group of 27 members used a consensus-driven modified Delphi approach to develop a set of global outcome measures for patients with OI. The International Classification of Functioning, Disability and Health (ICF), was used to define domains and organize the outcomes from the literature search. After reviewing the outcomes extracted from the literature, trials and registries, the working group agreed on a final selection of domains and their definition (ICF definition as well as a lay description). These domains were then presented to the focus groups who prioritized the outcome domains by taking into account the items important to the OI community. All content was collected and analyzed and final domains were determined. A consensus of appropriate measuring instruments for each domain was reached with Delphi rounds. The entire approach was in line with the International Consortium for Health Outcomes Measurement ICHOM methodology. Results More than 400 different outcome measures were identified in our literature search. After three Delphi rounds, 24 domains were selected. After the focus group sessions, the number of domains were reduced to 15. A consensus was reached on the measuring instruments to cover these domains for both children and adults. Conclusion The Key4OI project resulted in standard set of outcome measures focused on the needs and wishes of individuals with OI and their families. This outcome set will enable healthcare teams and systems to compare and to improve their care pathways and quality of care worldwide. Further studies are needed to evaluate the implementation of this standardized outcome set.

show abstract

“…However, it remains unclear which rating scale is the most appropriate for use in the Delphi phase of a COS development study. Qualitative interviews reported mixed feedback from user experience of different rating scales [ 13 ] and only one study has compared the use of two different rating scales, a 3-point and a 9-point scale, for rating preliminary outcomes [ 14 ]. The authors of this study reported that the use of the 9-point rating scale resulted in almost twice as many outcomes being rated as important compared with the 3-point rating scale in the first Delphi round.…”

Section: Introductionmentioning

confidence: 99%

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study

Remus

Smith

Wuytack

2021

BMC Med Res Methodol

View full text Add to dashboard Cite

Background As the development of core outcome sets (COS) increases, guidance for developing and reporting high-quality COS continues to evolve; however, a number of methodological uncertainties still remain. The objectives of this study were: (1) to explore the impact of including patient interviews in developing a COS, (2) to examine the impact of using a 5-point versus a 9-point rating scale during Delphi consensus methods on outcome selection and (3) to inform and contribute to COS development methodology by advancing the evidence base on COS development techniques. Methods Semi-structured patient interviews and a nested randomised controlled parallel group trial as part of the Pelvic Girdle Pain Core Outcome Set project (PGP-COS). Patient interviews, as an adjunct to a systematic review of outcomes reported in previous studies, were undertaken to identify preliminary outcomes for including in a Delphi consensus survey. In the Delphi survey, participants were randomised (1:1) to a 5-point or 9-point rating scale for rating the importance of the list of preliminary outcomes. Results Four of the eight patient interview derived outcomes were included in the preliminary COS, however, none of these outcomes were included in the final PGP-COS. The 5-point rating scale resulted in twice as many outcomes reaching consensus after the 3-round Delphi survey compared to the 9-point scale. Consensus on all five outcomes included in the final PGP-COS was achieved by participants allocated the 5-point rating scale, whereas consensus on four of these was achieved by those using the 9-point scale. Conclusions Using patient interviews to identify preliminary outcomes as an adjunct to conducting a systematic review of outcomes measured in the literature did not appear to influence outcome selection in developing the COS in this study. The use of different rating scales in a Delphi survey, however, did appear to impact on outcome selection. The 5-point scale demonstrated greater congruency than the 9-point scale with the outcomes included in the final PGP-COS. Future research to substantiate our findings and to explore the impact of other rating scales on outcome selection during COS development, however, is warranted.

show abstract

Participating in core outcome set development via Delphi surveys: qualitative interviews provide pointers to inform guidance

Cited by 21 publications

References 36 publications

Patient participation in Delphi surveys to develop core outcome sets: systematic review

Patient participation in Delphi surveys to develop core outcome sets: systematic review

A standard set of outcome measures for the comprehensive assessment of osteogenesis imperfecta

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study

Contact Info

Product

Resources

About