Unintended consequences: a qualitative study exploring the impact of collecting implementation process data with phone interviews on implementation activities
Abstract:Background
Qualitative data are crucial for capturing implementation processes, and thus necessary for understanding implementation trial outcomes. Typical methods for capturing such data include observations, focus groups, and interviews. Yet little consideration has been given to how such methods create interactions between researchers and study participants, which may affect participants’ engagement, and thus implementation activities and study outcomes. In the context of a clinical trial, w… Show more
“…The reflections were helpful in providing additional context to explain trial results, assessing fidelity and adaptations to the facilitation process, documenting facilitators’ perspectives on successes and challenges to implementation, and aiding facilitator recall during weekly updates to the CTAC team. Our findings align with others suggesting that regular check-ins during implementation may improve documentation of and engagement in implementation activities [ 34 ]. Additional research is needed to assess the potential of facilitator reflections as a novel data source to evaluate facilitation and implementation outcomes.…”
Background
Implementation facilitators support the adoption of evidence-based practices and other improvement efforts in complex healthcare settings. Facilitators are trained to develop essential facilitation skills and facilitator effectiveness is typically evaluated post-implementation, but little is known about how facilitators apply and adapt didactic knowledge after training, or how learning and refining experiential knowledge occurs during the facilitation process. We propose the use of reflective writing as a tool to document and support facilitator learning and facilitator effectiveness.
Methods
Using an instrumental case study of the Coordination Toolkit and Coaching (CTAC) project, we explore the use of reflective writing by facilitators to support their learning and effectiveness. Six primary care clinics participated in weekly hour-long facilitation calls over a 12-month period to implement quality improvement projects related to care coordination. Two facilitators completed templated reflections after each facilitation call for their assigned sites, totaling 269 reflections. We used the declarative-procedural-reflective model, which defines the process of skill development in clinical practice, to qualitatively analyze the reflections. Two independent coders used content analysis principles to code text that captured facilitators’ observations, evaluations, interpretations, and communication. Descriptive statistics were used to analyze reflections by facilitator and by code within and across reflections.
Results
CTAC facilitators primarily used the reflections to summarize the calls (observation), assess the facilitation process and the tasks and activities they used (evaluation), document their thoughts about how to improve their own effectiveness (interpretation), and describe their communication with implementing teams. Ninety-one percent of reflections included observations, 42% interpretation, 41% evaluation, and 44% facilitator communication. In total, we coded 677 segments of text within reflections: 39% represented observation, 20% interpretation, 18% evaluation, and 23% facilitator communication.
Conclusions
The process of reflective writing allowed the CTAC facilitators the time and structure to evaluate their facilitation and to think critically about how to adjust their facilitation in response to their observations and interpretations. Reflective writing is a feasible and acceptable tool to support and document facilitator learning and effectiveness.
Trial registration
The project was registered with ClinicalTrials.gov (NCT03063294) on February 24, 2017.
“…The reflections were helpful in providing additional context to explain trial results, assessing fidelity and adaptations to the facilitation process, documenting facilitators’ perspectives on successes and challenges to implementation, and aiding facilitator recall during weekly updates to the CTAC team. Our findings align with others suggesting that regular check-ins during implementation may improve documentation of and engagement in implementation activities [ 34 ]. Additional research is needed to assess the potential of facilitator reflections as a novel data source to evaluate facilitation and implementation outcomes.…”
Background
Implementation facilitators support the adoption of evidence-based practices and other improvement efforts in complex healthcare settings. Facilitators are trained to develop essential facilitation skills and facilitator effectiveness is typically evaluated post-implementation, but little is known about how facilitators apply and adapt didactic knowledge after training, or how learning and refining experiential knowledge occurs during the facilitation process. We propose the use of reflective writing as a tool to document and support facilitator learning and facilitator effectiveness.
Methods
Using an instrumental case study of the Coordination Toolkit and Coaching (CTAC) project, we explore the use of reflective writing by facilitators to support their learning and effectiveness. Six primary care clinics participated in weekly hour-long facilitation calls over a 12-month period to implement quality improvement projects related to care coordination. Two facilitators completed templated reflections after each facilitation call for their assigned sites, totaling 269 reflections. We used the declarative-procedural-reflective model, which defines the process of skill development in clinical practice, to qualitatively analyze the reflections. Two independent coders used content analysis principles to code text that captured facilitators’ observations, evaluations, interpretations, and communication. Descriptive statistics were used to analyze reflections by facilitator and by code within and across reflections.
Results
CTAC facilitators primarily used the reflections to summarize the calls (observation), assess the facilitation process and the tasks and activities they used (evaluation), document their thoughts about how to improve their own effectiveness (interpretation), and describe their communication with implementing teams. Ninety-one percent of reflections included observations, 42% interpretation, 41% evaluation, and 44% facilitator communication. In total, we coded 677 segments of text within reflections: 39% represented observation, 20% interpretation, 18% evaluation, and 23% facilitator communication.
Conclusions
The process of reflective writing allowed the CTAC facilitators the time and structure to evaluate their facilitation and to think critically about how to adjust their facilitation in response to their observations and interpretations. Reflective writing is a feasible and acceptable tool to support and document facilitator learning and effectiveness.
Trial registration
The project was registered with ClinicalTrials.gov (NCT03063294) on February 24, 2017.
Sheds for Life is a gender-specific tailored men's health initiative engaging “hard-to-reach” men in the Men's Shed setting in Ireland. It is implemented by multiple stakeholders at individual, provider, organization and systems level and thus multiple contextual factors influence its scalability. This research used established implementation science frameworks to guide participatory research approaches that captured the process and identified facilitators of and barriers to implementation and scale-up. Active recruitment, co-design processes, leadership and stakeholder engagement emerged as key facilitators of implementation. Prominent barriers were institutional capacity and funding. Acceptability, adoption and appropriateness of the initiative were high among stakeholders with sustainability largely contingent on funding and staff resources. Findings make a valuable contribution to knowledge by capturing the process involved in the implementation of a complex multi-level men's health intervention. It provides a “how to” guide of strategies to engage hard-to-reach men with health promotion, the operationalization and application of implementation frameworks in community-based health promotion, and the implementation of health promotion in Men's Sheds. Documented barriers and facilitators that impact implementation of a community-based men's health program are rare and provide a valuable blueprint for practitioners, researchers and policy makers in the field.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.