Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review

Abstract

ObjectiveDelphi technique is a structured process commonly used to developed healthcare quality indicators, but there is a little recommendation for researchers who wish to use it. This study aimed 1) to describe reporting of the Delphi method to develop quality indicators, 2) to discuss specific methodological skills for quality indicators selection 3) to give guidance about this practice.Methodology and Main FindingThree electronic data bases were searched over a 30 years period (1978–2009). All articles that used the Delphi method to select quality indicators were identified. A standardized data extraction form was developed. Four domains (questionnaire preparation, expert panel, progress of the survey and Delphi results) were assessed. Of 80 included studies, quality of reporting varied significantly between items (9% for year's number of experience of the experts to 98% for the type of Delphi used). Reporting of methodological aspects needed to evaluate the reliability of the survey was insufficient: only 39% (31/80) of studies reported response rates for all rounds, 60% (48/80) that feedback was given between rounds, 77% (62/80) the method used to achieve consensus and 57% (48/80) listed quality indicators selected at the end of the survey. A modified Delphi procedure was used in 49/78 (63%) with a physical meeting of the panel members, usually between Delphi rounds. Median number of panel members was 17(Q1:11; Q3:31). In 40/70 (57%) studies, the panel included multiple stakeholders, who were healthcare professionals in 95% (38/40) of cases. Among 75 studies describing criteria to select quality indicators, 28 (37%) used validity and 17(23%) feasibility.ConclusionThe use and reporting of the Delphi method for quality indicators selection need to be improved. We provide some guidance to the investigators to improve the using and reporting of the method in future surveys.

Cognitive Aids in Medicine Assessment Tool (CMAT): preliminary validation of a novel tool for the assessment of emergency cognitive aids

“…We employed a modified Delphi approach [12], drawing on our own expertise as anaesthetists with an academic interest in human factors, to develop the new medical tool.…”

Controversy between anesthesiologists and obstetricians on the labour ward: the Delphi method is used as a consensus-building technique

“…The technique is designed as a group exercise that aims to achieve a convergence of opinion that may prove very useful. [8][9][10][11][15][16][17][18][19] The interactions of experts may lead to a reduction in individual bias. The Delphi method is a wellrecognized tool for solving problems in healthcare, and it is increasingly being used to reach consensus around many topics in medicine, including education, development of clinical guidelines, and prioritization of research topics.…”

“…The Delphi method is a wellrecognized tool for solving problems in healthcare, and it is increasingly being used to reach consensus around many topics in medicine, including education, development of clinical guidelines, and prioritization of research topics. 16,17 It has been argued that the forecasting accuracy of the Delphi technique is clearly reliable. For example, a Delphi study involving medical doctors evaluating the forecasting application of the method revealed that, in 75% of cases, the estimated values proved to be less than 10% different from the values observed.…”

Content Validation of a Novel Screening Tool to Identify Emergency Department Patients With Significant Palliative Care Needs

“…While no consensus recommendations exist for the structure of a modified Delphi panel, we modeled the structure of our Delphi technique on the recommendations reported by Boulkedid et al 65 in their 2011 systematic review of the topic, including: 1) the use of a preset level of percent agreement and stop criteria, 2) use of a 1 to 9 Likert rating scale, 3) two to three rounds, 4) heterogeneous panel, 5) and mixed qualitative and quantitative feedback.…”

“…98 The size of the expert panel was based on recommended Delphi methodology and is similar in size to other Delphi panels described in the literature. 65 However, while a smaller panel is more advantageous in terms of discussion and consensus, the results may be limited by skewed sampling of experts. 99 An attempt was made to diminish this effect by inviting a diverse group of expert panelists to participate, in terms of years in practice, education level, sex, practice type and setting, and geographic location.…”