Delphi procedure in core outcome set development: rating scale and consensus criteria determined outcome selection

Meyer, Dorien De; Kottner, Jan; Beele, Hilde; Schmitt, Jochen; Lange, Toni; Hecke, Ann Van; Verhaeghe, Sofie; Beeckman, Dimitri

doi:10.1016/j.jclinepi.2019.03.011

Cited by 57 publications

(50 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In our study, the 5-point scale resulted in twice as many outcomes in the preliminary PGP-COS after three rounds compared with the 9-point scale. These results contradict the findings of twice as many outcomes selected as "critical" on the 9-point scale compared to the 3-point scale reported by De Meyer and colleagues [11]. Although we cannot directly compare our results as scale differences were only explored in the first round of de Meyer's Delphi and our consensus definitions differed, it is plausible that the sizeable difference in rating options between a 3-point and 9-point compared to a 5-point and 9-point may explain the conflicting results between our studies.…”

Section: Impact Of Survey Scalescontrasting

confidence: 99%

“…The COMET (Core Outcome Measures in Effectiveness Trials) Initiative provides guidance for using the Delphi technique to prioritise outcomes in developing a COS, but recognises also that there are a number of methodological uncertainties surrounding this method which need to be further explored [2,4,10]. For example, a variety of Likert scale scoring systems exist for use in COS development; however, it remains unclear which rating scale is the most appropriate for use in the Delphi phase of a COS development study, with one other study only identified that compared the use of two different Likert scales, a 3-point and a 9-point scale, for rating preliminary outcomes [11]. The authors of this study reported that the use of the 9-point Likert scale resulted in almost twice as many outcomes being rated as important compared with the 3-point Likert scale in the first Delphi round, but they did not, however, analyse how each scale impacted on the final COS, and the impact of using different rating scales on outcome selection, and the actual COS remains unclear.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study.

Remus

Smith

Wuytack

2020

Preprint

View full text Add to dashboard Cite

Background As the development of core outcome sets (COS) increases, guidance for developing and reporting high-quality COS continues to evolve; however, a number of methodological uncertainties still remain. The objectives of this study were: (1) to explore the impact of including patient interviews in developing a COS (2) to examine the impact of using a 5-point versus a 9-point rating scale during Delphi consensus methods on outcome selection and the COS and (3) to inform and contribute to COS development methodology by advancing the evidence base on COS development techniques. Methods Semi-structured patient interviews (i) and a nested randomised controlled parallel group trial (ii) as part of the Pelvic Girdle Pain Core Outcome Set project (PGP-COS). Patient interviews, as an adjunct to a systematic review of outcomes reported in previous studies, were undertaken to identify preliminary outcomes for including in a Delphi consensus survey. In the Delphi survey, participants were randomised (1:1) to a 5-point or 9-point Likert scale for rating the importance of the list of preliminary outcomes. Results None of the outcomes identified from the patient interviews only were included in the final PGP-COS. The 5-point rating scale resulted in twice as many outcomes reaching consensus after the 3-round Delphi survey compared to the 9-point scale. Consensus on all five outcomes included in the final PGP-COS was achieved by participants allocated the 5-point rating scale, whereas consensus on four of these was achieved by those using the 9-point scale. Conclusions Using patient interviews to identify preliminary outcomes as an adjunct to conducting a systematic review of outcomes measured in the literature, does not appear to influence outcome selection in developing a COS. The use of different rating scales in a Delphi survey, however, does appear to impact on outcome selection. The 5-point scale demonstrated greater congruency than the 9-point scale with the outcomes included in the final PGP-COS. Use of a 5-point scale can thus be considered for Delphi surveys in COS development studies. Future research to explore the impact of other rating scales on outcome selection during COS development is still warranted.

show abstract

Section: Impact Of Survey Scalescontrasting

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study.

Remus

Smith

Wuytack

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In our study, the 5-point scale resulted in twice as many outcomes in the preliminary PGP-COS after three rounds compared with the 9-point scale. These results contradict the ndings of twice as many outcomes selected as "critical" on the 9-point scale compared to the 3-point scale reported by De Meyer and colleagues [14]. Although we cannot directly compare our results as scale differences were only explored in the rst round of de Meyer's Delphi and our consensus de nitions differed, it is plausible that the sizeable difference in rating options between a 3-point and 9-point compared to a 5-point and 9-point may explain the con icting results between our studies.…”

Section: Impact Of Survey Scalescontrasting

confidence: 96%

“…However, it remains unclear which rating scale is the most appropriate for use in the Delphi phase of a COS development study. Qualitative interviews reported mixed feedback from user experience of different rating scales [13] and only one study has compared the use of two different rating scales, a 3-point and a 9-point scale, for rating preliminary outcomes [14]. The authors of this study reported that the use of the 9-point rating scale resulted in almost twice as many outcomes being rated as important compared with the 3-point rating scale in the rst Delphi round.…”

mentioning

confidence: 99%

“…Too many outcomes after each Delphi round is challenging because the goal of this process is to narrow down a larger list into a minimum set and a COS with too many outcomes may not be feasible or may not be ultimately adopted in research and clinical practice. Additionally, the authors of this study did not analyse how each scale impacted the nal COS, so the impact of using different rating scales on outcome selection and the nal COS remains unknown and needs to be explored further [14]. For this reason, we embedded a randomised trial within our Pelvic Girdle Pain Core Outcome Set (PGP-COS) development project to compare the impact of a 5-point versus a 9-point rating scale on preliminary outcome selection and the nal agreed COS [10,15,16].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study.

Remus

Smith

Wuytack

2020

Preprint

View full text Add to dashboard Cite

Background: As the development of core outcome sets (COS) increases, guidance for developing and reporting high-quality COS continues to evolve; however, a number of methodological uncertainties still remain. The objectives of this study were: (1) to explore the impact of including patient interviews in developing a COS, (2) to examine the impact of using a 5-point versus a 9-point rating scale during Delphi consensus methods on outcome selection and the COS and (3) to inform and contribute to COS development methodology by advancing the evidence base on COS development techniques.Methods: Semi-structured patient interviews and a nested randomised controlled parallel group trial as part of the Pelvic Girdle Pain Core Outcome Set project (PGP-COS). Patient interviews, as an adjunct to a systematic review of outcomes reported in previous studies, were undertaken to identify preliminary outcomes for including in a Delphi consensus survey. In the Delphi survey, participants were randomised (1:1) to a 5-point or 9-point scale for rating the importance of the list of preliminary outcomes.Results: None of the outcomes identified from the patient interviews were included in the final PGP-COS. The 5-point rating scale resulted in twice as many outcomes reaching consensus after the 3-round Delphi survey compared to the 9-point scale. Consensus on all five outcomes included in the final PGP-COS was achieved by participants allocated the 5-point rating scale, whereas consensus on four of these was achieved by those using the 9-point scale.Conclusions: Using patient interviews to identify preliminary outcomes as an adjunct to conducting a systematic review of outcomes measured in the literature does not appear to influence outcome selection in developing a COS. The use of different rating scales in a Delphi survey, however, does appear to impact on outcome selection. The 5-point scale demonstrated greater congruency than the 9-point scale with the outcomes included in the final PGP-COS. Use of a 5-point scale might be considered for Delphi surveys in COS development studies. Future research to substantiate our findings and to explore the impact of other rating scales on outcome selection during COS development, however, is still warranted.

show abstract

Recommendations for Medical and Mental Health Care in Assisted Living Based on an Expert Delphi Consensus Panel

Zimmerman

Sloane

Wretman³

et al. 2022

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceAssisted living (AL) is the largest provider of residential long-term care in the US, and the morbidity of AL residents has been rising. However, AL is not a health care setting, and concern has been growing about residents’ medical and mental health needs. No guidance exists to inform this care.ObjectiveTo identify consensus recommendations for medical and mental health care in AL and determine whether they are pragmatic.Evidence ReviewA Delphi consensus statement study was conducted in 2021; as a separate effort, the extent to which the recommendations are reflected in practice was examined in data obtained from 2016 to 2021 (prepandemic). In the separate effort, data were from a 7-state study (Arkansas, Louisiana, New Jersey, New York, Oklahoma, Pennsylvania, Texas). The 19 Delphi panelists constituted nationally recognized experts in medical, nursing, and mental health needs of and care for older adults; dementia care; and AL and long-term care management, advocacy, regulation, and education. One invitee was unavailable and nominated an alternate. The primary outcome was identification of recommended practices based on consensus ratings of importance. Panelists rated 183 items regarding importance to care quality and feasibility.FindingsConsensus identified 43 recommendations in the areas of staff and staff training, nursing and related services, resident assessment and care planning, policies and practices, and medical and mental health clinicians and care. To determine the pragmatism of the recommendations, their prevalence was examined in the 7-state study and found that most were in practice. The items reflected the tenets of AL, the role of AL in providing dementia care, the need for pragmatism due to the diversity of AL, and workforce needs.Conclusions and RelevanceIn this consensus statement, 43 recommendations important to medical and mental health care in AL were delineated that are highly pragmatic as a guide for practice and policy.

show abstract

Delphi procedure in core outcome set development: rating scale and consensus criteria determined outcome selection

Cited by 57 publications

References 32 publications

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study.

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study.

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study.

Recommendations for Medical and Mental Health Care in Assisted Living Based on an Expert Delphi Consensus Panel

Contact Info

Product

Resources

About