Expect the Unexpected: A Qualitative Study of the Ripple Effects of Children’s Mental Health Services Implementation Efforts

Abstract

Background: Strategies to implement evidence-based interventions (EBIs) in children’s mental health services have complex direct and indirect causal impacts on multiple outcomes. Ripple effects are outcomes that are caused by EBI implementation efforts and are unplanned, unanticipated, and/or more salient to stakeholders other than to researchers and implementers. The purpose of the current paper is to provide a compilation of possible ripple effects associated with EBI implementation strategies in children’s mental health services, to be used for treatment development, implementation planning, research, and quality improvement. Methods: Participants were identified via expert nomination and snowball sampling. Online surveys were completed by 81 participants who contributed ripple effects, each representing one of five roles: providers of mental health services to children or youth, researchers, policy makers, caregivers, and youth. A partially directed conventional content analysis with consensus decision making was used to code ripple effects. Results: Four hundred and four unique responses were coded into 66 ripple effects and 14 categories. Example categories include general knowledge, skills, attitudes, and confidence about using EBIs; general job-related ripple effects; EBI treatment adherence, fidelity, and alignment; gaming the system; equity and stigma; shifting roles, role clarity, and task shifting; economic costs and benefits; EBI treatment availability, access, participation, attendance, barriers, and facilitators; clinical process and treatment quality; client engagement, therapeutic alliance, and client satisfaction; clinical organization structure, relationships in the organization, process, and functioning; youth client and caregiver outcomes; and use of EBI strategies and insights in one’s own life. Conclusions: This research advances the field by providing children’s mental health implementers, researchers, funders, policy makers, and consumers with a menu of potential ripple effects for consideration when evaluating implementation quality and impact. It can be a practical tool to ensure compliance with the guidance of experts from Quality Improvement/Quality Assurance, Complexity Science, Diffusion of Innovation Theory, and the Medical Research Council. Future phases will match potential ripple effects with the most salient children’s mental health implementation strategies for each participant role. Trial Registration: None.

How do oilcloth sessions work? A realist evaluation approach to exploring ripple effects in an implementation strategy

“…Although it is recognized that unintended outcomes of social programs might affect the development, implementation, and evaluation of public health interventions (Bonell et al, 2015), these are rarely addressed in the literature (Lorenc and Oliver, 2014) and implementation research. Lorenc and Oliver (2014) Focusing on unintended outcomes in implementation research could help us anticipate negative unintended outcomes or identify unintended consequences that could be leveraged or addressed in future implementation efforts to enhance outcomes, effectiveness and sustainability (Bonell et al, 2015;Pullmann et al, 2022). There is a need within implementation research to incorporate attention to unintended outcomes to fully understanding the impact of implementation strategies.…”

“…Implementation strategies resemble public health interventions in the sense that both concern activities of change in complex systems based on underlying assumptions of feasibility and effectiveness. Focusing on unintended outcomes in implementation research could help us anticipate negative unintended outcomes or identify unintended consequences that could be leveraged or addressed in future implementation efforts to enhance outcomes, effectiveness and sustainability (Bonell et al ., 2015; Pullmann et al ., 2022). There is a need within implementation research to incorporate attention to unintended outcomes to fully understanding the impact of implementation strategies.…”

“…In this study, we define ripple effects as unintentional, unplanned, unanticipated outcomes that are caused by an implementation strategy (Pullmann et al ., 2022). However, in some realist evaluation studies, the concept of ripple effects is applied to illustrate how intervention activities generate series of (sequential) CMO configurations (Jagosh et al ., 2015).…”

“…Attention to unintended outcomes of implementation strategies may help implementation researchers rethink the activities involved to reduce negative unintended outcomes or leverage positive unintended consequences to enhance implementation and health outcomes. We suggest future studies within implementation research incorporate attention to unintended ripple effects and outcomes to fully understand the impact of implementation strategies (Pullmann et al ., 2022)…”

“…However, implementation strategies are social processes that require behavioral changes (Neal and Neal, 2019). This may lead to unplanned, non-linear, or unexpected “ripple effects” (Lipsitz, 2012; Pullmann et al ., 2022). Ripple effects have been discussed in the literature as being positive or negative outcomes that are caused by implementation strategies and are unintentional, unplanned, unanticipated, and/or more salient to stakeholders other than implementers and researchers (Pullmann et al ., 2022).…”