Development of an Implementation Process Model: a Delphi study

Abstract

Background There is general scarcity of research on key elements of implementation processes and the factors which impact implementation success. Implementation of healthcare interventions is a complex process. Tools to support implementation can facilitate this process and improve effectiveness of the interventions and clinical outcomes. Understanding the impact of implementation support tools is a critical aspect of this process. The objective of this study was to solicit knowledge and agreement from relevant implementation science and knowledge translation healthcare experts in order to develop a process model of key elements in the implementation process. Methods A two round, modified Delphi study involving international experts in knowledge translation and implementation (researchers, scientists, professors, decision-makers) was conducted. Participants rated and commented on all aspects of the process model, including the organization, content, scope, and structure. Delphi questions rated at 75% agreement or lower were reviewed and revised. Qualitative comments supported the restructuring and refinement. A second-round survey followed the same process as Round 1. Results Fifty-four experts participated in Round 1, and 32 experts participated in Round 2. Twelve percent (n = 6) of the Round 1 questions did not reach agreement. Key themes for revision and refinement were: stakeholder engagement throughout the process, iterative nature of the implementation process; importance of context; and importance of using guiding theories or frameworks. The process model was revised and refined based on the quantitative and qualitative data and reassessed by the experts in Round 2. Agreement was achieved on all items in Round 2 and the Delphi concluded. Additional feedback was obtained regarding terminology, target users and definition of the implementation process. Conclusions High levels of agreement were attained for all sub-domains, elements, and sub-elements of the Implementation Process Model. This model will be used to develop an Implementation Support Tool to be used by healthcare providers to facilitate effective implementation and improved clinical outcomes.

Identification of quality indicators of public health nursing practice: “modified Delphi” approach

“…Research suggests that a minimum panel of 15−20 experts is recommended to ensure sufficient contributions in a Delphi (Keeney et al, 2011;Parker et al, 2021). Where expert panels are homogeneous, 10-15 members have been identified as an appropriate number (Delbecq et al, 1975;Keeney et al, 2011).…”

“…Accounting for the commonly high attrition rate in Delphi studies, the recruitment target for this study was set at 20 participants. This also allowed for inclusion of disparate views (Parker et al, 2021). The inclusion criteria identified 34 eligible participants all of whom were invited to participate.…”

“…The number of experts was 23, some Delphi studies have employed over 60 participants to provide representative information (Hunter et al, 2021;O'Connor et al, 2020) while others have employed as few as nine to fifteen (Basger et al, 2012;Campbell et al, 2011;Fitch et al, 2001). Nonetheless, participants were recruited using purposeful and snowball sampling ensuring geographical representation, a mix of national and international expertise, enabling a robust approach (Keeney et al, 2011), and a high level and diverse range of expertise (Parker et al, 2021).…”

“…The anonymity in a Delphi study is both a strength and limitation, it helps to reduce the influence of participants who may dominate an in-person session but also eliminates the opportunity for the discussion and discovery that can occur during in-person meetings (Keeney et al, 2011;Parker et al, 2021). Bias in the selection of indicators presented to the participants could be suggested, however, we minimized this through an extensive literature search, incorporation of the voices of mothers and PHNs from the initial exploratory phase and we provided participants the opportunity to add indicator topics.…”

“…Using Delphi techniques such as the RAND/UCLA appropriateness method, which combine expert opinion, available evidence, and indicators based on clinical guidelines should be utilized (Fitch et al, 2001). Moreover, using this technique ensures face and content validity of the developed quality indicators (Miller et al, 2020;Parker et al, 2021;Power et al, 2015). Helsloot et al (2017) used a "modified Delphi" study to identify quality indicators for postnatal care in Finland.…”