Tool for evaluating research implementation challenges: A sense-making protocol for addressing implementation challenges in complex research settings

Abstract

BackgroundMany challenges arise in complex organizational interventions that threaten research integrity. This article describes a Tool for Evaluating Research Implementation Challenges (TECH), developed using a complexity science framework to assist research teams in assessing and managing these challenges.MethodsDuring the implementation of a multi-site, randomized controlled trial (RCT) of organizational interventions to reduce resident falls in eight nursing homes, we inductively developed, and later codified the TECH. The TECH was developed through processes that emerged from interactions among research team members and nursing home staff participants, including a purposive use of complexity science principles.ResultsThe TECH provided a structure to assess challenges systematically, consider their potential impact on intervention feasibility and fidelity, and determine actions to take. We codified the process into an algorithm that can be adopted or adapted for other research projects. We present selected examples of the use of the TECH that are relevant to many complex interventions.ConclusionsComplexity theory provides a useful lens through which research procedures can be developed to address implementation challenges that emerge from complex organizations and research designs. Sense-making is a group process in which diverse members interpret challenges when available information is ambiguous; the groups’ interpretations provide cues for taking action. Sense-making facilitates the creation of safe environments for generating innovative solutions that balance research integrity and practical issues. The challenges encountered during implementation of complex interventions are often unpredictable; however, adoption of a systematic process will allow investigators to address them in a consistent yet flexible manner, protecting fidelity. Research integrity is also protected by allowing for appropriate adaptations to intervention protocols that preserve the feasibility of ‘real world’ interventions.

More-2-Eat: evaluation protocol of a multi-site implementation of the Integrated Nutrition Pathway for Acute Care

“…The key informants and focus group participants reviewed an information letter and provided written consent before the discussion began. These qualitative data will be used to understand the setting and context, including suggestions for making improvements in nutrition care within the unit [55,65]. Both focus groups and interviews were conducted by one researcher to promote consistency.…”

“…The principal outcome of this work will be the development of a toolkit that provides strategies, tips, resources and tools for others to use when implementing INPAC. Healthcare is considered a complex adaptive system and thus participatory action research is appropriate for evaluating an implementation program that relies on change within this system [51,[53][54][55]. Process and outcome evaluation will also address the following research questions: …”

O pensamento complexo e suas repercussões na gestão em enfermagem e saúde

“…Assim, a partir do processo de leitura e análise, identificou-se que o conceito mais substancialmente abordado dentro dessa perspectiva foi o dos sistemas complexos adaptativos (14,15). Dentre os ambientes definidos como tal, pontuam-se a atenção primária (14), os lares de idosos, os hospitais, os consultórios médicos e as clínicas (13,15,16).…”

Commentary on Pardavila-Belioet al. (2015): Reporting on randomized control trials or giving details on how the cake was made-the list of ingredients and the implementation process

“…As some authors have noted: 'the interaction of the intervention, users, and context of practice determines the rate and extent of adoption ' [8]. In recent years, the concern on the implementation quality has increased and there are tools and guidelines available [9][10][11].A previous review of multi-component interventions to prevent and control tobacco use among college students highlighted the extent to which the programme components are being implemented, and how institutions are integrating programme components into comprehensive efforts [12]. In the Spanish college trial there were also several deviations from the protocol, as the authors noted.…”

“…As some authors have noted: 'the interaction of the intervention, users, and context of practice determines the rate and extent of adoption ' [8]. In recent years, the concern on the implementation quality has increased and there are tools and guidelines available [9][10][11].…”