Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study

Abstract

Background As the development of core outcome sets (COS) increases, guidance for developing and reporting high-quality COS continues to evolve; however, a number of methodological uncertainties still remain. The objectives of this study were: (1) to explore the impact of including patient interviews in developing a COS, (2) to examine the impact of using a 5-point versus a 9-point rating scale during Delphi consensus methods on outcome selection and (3) to inform and contribute to COS development methodology by advancing the evidence base on COS development techniques. Methods Semi-structured patient interviews and a nested randomised controlled parallel group trial as part of the Pelvic Girdle Pain Core Outcome Set project (PGP-COS). Patient interviews, as an adjunct to a systematic review of outcomes reported in previous studies, were undertaken to identify preliminary outcomes for including in a Delphi consensus survey. In the Delphi survey, participants were randomised (1:1) to a 5-point or 9-point rating scale for rating the importance of the list of preliminary outcomes. Results Four of the eight patient interview derived outcomes were included in the preliminary COS, however, none of these outcomes were included in the final PGP-COS. The 5-point rating scale resulted in twice as many outcomes reaching consensus after the 3-round Delphi survey compared to the 9-point scale. Consensus on all five outcomes included in the final PGP-COS was achieved by participants allocated the 5-point rating scale, whereas consensus on four of these was achieved by those using the 9-point scale. Conclusions Using patient interviews to identify preliminary outcomes as an adjunct to conducting a systematic review of outcomes measured in the literature did not appear to influence outcome selection in developing the COS in this study. The use of different rating scales in a Delphi survey, however, did appear to impact on outcome selection. The 5-point scale demonstrated greater congruency than the 9-point scale with the outcomes included in the final PGP-COS. Future research to substantiate our findings and to explore the impact of other rating scales on outcome selection during COS development, however, is warranted.

A core outcome set for research and clinical practice in women with pelvic girdle pain: PGP-COS

“…There is currently no consensus on which rating scale should be used in COS development and, as such, different scales have been used in previous COS. The results of the embedded methodological study are reported elsewhere [24]. During the Delphi study, however, any outcome rated as important on either scale (Table 2), as per our consensus definition detailed below, was forwarded to the subsequent round.…”

Development of a core outcome set and outcome definitions for studies on uterus-sparing treatments of adenomyosis (COSAR): an international multistakeholder-modified Delphi consensus study

“…Each item was graded from 1 to 9 ( De Meyer et al , 2019 ), with the additional option “I can’t rate the outcome because I don’t know the outcome”. Written anchors were provided to reduce measurement error ( Beckstead, 2014 ; Remus et al , 2021 ) (1. Extremely unimportant; 2.…”

Strategies for implementing pet robots in care homes and nursing homes for residents with dementia: protocol for a modified Delphi study

“…In the third section, participants will be presented with statements and be invited to rate the importance of the implementation strategies. Whilst there is no gold standard for selecting an appropriate scale for consensus processes to identify implementation strategies [ 55 ], 9-point scales have been recommended by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group to assess the importance of research evidence [ 56 ] and have been suggested to have more discriminatory power than other scales [ 57 ]. As such, a 9-point Likert scale will be used.…”