Using consensus group methods such as Delphi and Nominal Group in medical education research

Abstract

Consensus group methods are widely used in research to identify and measure areas where incomplete evidence exists for decision-making. Despite their widespread use, these methods are often inconsistently used and reported. Using examples from the three most commonly used methods, the Delphi, Nominal Group and RAND/UCLA; this paper and associated Guide aim to describe these methods and to highlight common weaknesses in methodology and reporting. The paper outlines a series of recommendations to assist researchers using consensus group methods in providing a comprehensive description and justification of the steps taken in their study.

Rediscovering the Delphi Technique: A Review of the Literature

“…Norman Dalkey introduced the Delphi technique around 1940-1950 for a RAND corporation military defense project "Project Delphi" concerning the decision-making for atomic bombing (Humphrey-Murto et al, 2017;Plessus & Human, 2007;Skulmoski et al, 2007). Olaf Helmer and Norman Dalkey pioneered and validated the Delphi technique as a scientifically rigorous research strategy with a statistical significance that was more accurate than group or individual opinions (Plessus & Human, 2007).…”

“…The experts reviewed how respective feedback aligned with other group members and then it allowed them to change their judgments if desired. The collaborative work continued over a series of iterative rounds until consensus and stability (majority agreement often called majority ruling) were reached (Humphrey-Murto, 2017;Kennedy, 2004).…”

“…The first step in the process is to select the topic and the questionnaire (Tack et al, 2017). Questions in the survey may be derived from literature review or after consulting with experts in the field; therefore, clarity in wording is critical (Humphrey-Murto et al, 2017). An initial questionnaire may be piloted in a small group that will be excluded from the main study (Humphrey-Murto et al, 2017;Staykova, 2012).…”

Medical students' attitudes towards the use of virtual patients

“…When assessing learning tools and no instruments are available, one must create one's own instruments (Sullivan, 2011) to assess the educational tool. In order to correctly and most reliably develop a method of attitude assessment, expert panels are recommended (Humphrey-Murto, Varpio, Gonsalves, & Wood, 2016). Downing points out that we have moved from the traditional type of scale validity towards a definition of construct validity as the only appropriate measurement of validity (Downing, 2003).…”

When I say … consensus group methods

“…We contend that CGMs are valuable tools in the medical educator's scholarship toolkit and are worthy of careful methodological critique and development. Furthermore, just as criteria for conducting randomised controlled trials are now established, guidelines for conducting CGMs have been suggested . When the ontological and epistemological roots of CGMs are clearly articulated and the techniques by which they are applied are better vetted, a shared understanding of what it means when we say ‘consensus group methods’ will be achieved.…”