A critical review of the Delphi technique as a research methodology for nursing

Keeney, Sinead; Hasson, Felicity; McKenna, Hugh

doi:10.1016/s0020-7489(00)00044-4

Cited by 989 publications

(1,094 citation statements)

References 30 publications

Supporting

Mentioning

1,053

Contrasting

Unclassified

Order By: Relevance

“…21 This technique forces participants to reach a consensus and does not allow conversation to discuss issues. Therefore, one relies on the 'expertise' of the participants.…”

Section: Discussionmentioning

confidence: 99%

Simulation performance checklist generation using the Delphi technique

Morgan

Lam-McCulloch

Herold-McIlroy

et al. 2007

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

Purpose: Performance assessment using high fidelity simulation is problematic, due to the difficulty in developing valid and reliable evaluation tools. The Delphi technique is a consensus based content generation method used for multiple purposes such as policy development, best-evidence practice guidelines and competency assessments. The purpose of this study was to develop checklists using a modified Delphi technique to evaluate the performance of practicing anesthesiologists managing two simulated scenarios. Methods:The templates for two simulation scenarios were emailed to five anesthesiologists who were asked to generate performance items. Data were collated anonymously and returned. An a priori decision was made to delete items endorsed by ≤ 20% of participants. This process of collection, collation and re-evaluation was repeated until consensus was reached. Four independent raters used the checklist to assess three subjects managing the two simulation scenarios. Interrater reliability was assessed using average measures intraclass correlation (ICC) and repeated measures analysis of variance (ANOVA) was used to assess differences in difficulty between scenarios. Results:The final checklists included 131 items for scenario 1 and 126 items for scenario 2. The mean inter-rater reliability was 0.921 for scenario 1 and 0.903 for scenario 2. Repeated measures ANOVA revealed no statistically significant difference in difficulty between scenarios.Discussion: The Delphi technique can be very useful to generate consensus based evaluation tools with high content and face validity compared to subjective evaluative tools. Since there was no difference in scenario difficulty, these scenarios can be used to determine the effect of educational interventions on performance.

show abstract

“…21 This technique forces participants to reach a consensus and does not allow conversation to discuss issues. Therefore, one relies on the 'expertise' of the participants.…”

Section: Discussionmentioning

confidence: 99%

Simulation performance checklist generation using the Delphi technique

Morgan

Lam-McCulloch

Herold-McIlroy

et al. 2007

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

show abstract

“…This method has been successfully followed in radiotherapy research as well as other research fields. [36][37][38][39][40][41][42] Since no patients were involved in our surveys, the law in the Netherlands does not require ethics clearance. Figure 1.…”

Section: Methodsmentioning

confidence: 99%

How to measure innovation in radiotherapy: an application of the Delphi method

Jacobs

Boersma

Dekker

et al. 2015

JHA

View full text Add to dashboard Cite

Objective: Innovation is an important driver for improving the quality of health care, yet a tension exists between innovation and providing cost-effective health care. To develop strategies that promote innovation, parameters are needed that are indicative of innovation. However, no recognised indicators of innovation in radiotherapy are currently available. The aim of this study is to fill that gap by providing a framework for measuring innovation. This should facilitate future multi-centric studies on strategies aimed at promoting innovation in radiotherapy. Methods: We applied the Delphi method in four rounds. The chairpersons of all Dutch radiotherapy departments were asked to suggest indicators. The resulting inventory was assessed by a number of Dutch radiation oncologists, medical physicists and managers. After implementig a cut-off score on suitability and measurability, we asked Dutch professors on innovation to assess the remaining indicators. Finally, the chairpersons reached consensus. Results: On the basis of the Delphi study, we derived 13 indicators in four categories, more specific product innovation, technology innovation, market innovation and organisational innovation, for measuring both incremental and radical innovations in radiotherapy; these indicators are also suitable for measuring the generation and adoption of innovations. Conclusions: We were successful in reaching consensus amongst the experts on indicators that measure innovations in radiotherapy. The developed tool will be used to investigate the relation between innovation and possible factors inhibiting or stimulating successful innovation and between the level of innovation and its effects.

show abstract

“…The method has many similarities with a Delphi approach, which is frequently used in the human health-care sector (e.g. Keeney et al, 2001) but which has also been applied within animal husbandry and management, for a quantification of expert knowledge (Van der Fels-Klerx et al, 2000;Sørensen et al, 2002).…”

Section: Methodsmentioning

confidence: 99%

Developing a HACCP-like system for improving animal health and welfare in organic egg production – based on an expert panel analysis

Hegelund

Sørensen

2007

Animal

View full text Add to dashboard Cite

In the process of developing a generic Hazard Analysis and Critical Control Point (HACCP)-like system for securing animal health and welfare in organic egg production, an expert panel analysis was used to perform the initial hazard analysis. Eighteen advisers and researchers in organic egg production were included in the expert panel. In a series of four questionnaires, the expert panel first scored 34 health and welfare problems seen in Danish organic egg production. Based on scorings of severity and occurrence, 10 problems were selected for further analysis. The experts subsequently suggested and scored risk factors for those problems and finally suggested control points, alarm values indicating the need for corrective actions in order to control the risk factors and monitoring frequencies of these. The 10 selected problems were hunger, thirst, piling, crop impaction, blackhead, pasteurellosis, bone fractures, cannibalism, predators and red mites. A total of 154 different risk factors were suggested for these problems. The 41 risk factors which rated highest in a combined scoring of importance and occurrence were selected for further analysis. There was a high degree of consensus between experts when scoring both problems and risk factors. The level of consensus, as defined by an interquartile range <1, was 79% to 100% when scoring the health and welfare problems (scale 1-5) and 77% to 95% when scoring the risk factors (scale 1-4). On average, 5.8 control points were suggested for every risk factor. Alarm values were often not detailed enough to be of practical significance and further analysis is needed in order to define these. The experts were highly diverse in their suggested monitoring frequencies and establishment of monitoring schemes should be part of developing the farm specific systems. An expert panel analysis based on questionnaires was a useful tool during the first steps of developing a HACCP plan, conducting a hazard analysis and suggesting control points. However, care should be taken regarding the panel's size and fields of expertise in order to assure that the panel fully covers the field of interest throughout the study. A further development of the hazard analysis into a farm-specific HACCP system would include assembling an on-farm HACCP team consisting of farmers and advisors for the practical application of control points, alarm values and monitoring schemes. The results from the fourth questionnaire would be useful as input for this work.

show abstract

A critical review of the Delphi technique as a research methodology for nursing

Cited by 989 publications

References 30 publications

Simulation performance checklist generation using the Delphi technique

Simulation performance checklist generation using the Delphi technique

How to measure innovation in radiotherapy: an application of the Delphi method

Developing a HACCP-like system for improving animal health and welfare in organic egg production – based on an expert panel analysis

Contact Info

Product

Resources

About