Research design issues for evaluating complex multicomponent interventions in neighborhoods and communities

Abstract

Major advances in population health will not occur unless we translate existing knowledge into effective multicomponent interventions, implement and maintain these in communities, and develop rigorous translational research and evaluation methods to ensure continual improvement and sustainability. We discuss challenges and offer approaches to evaluation that are key for translational research stages 3 to 5 to advance optimized adoption, implementation, and maintenance of effective and replicable multicomponent strategies. The major challenges we discuss concern (a) multiple contexts of evaluation/research, (b) complexity of packages of interventions, and (c) phases of evaluation/research questions. We suggest multiple alternative research designs that maintain rigor but accommodate these challenges and highlight the need for measurement systems. Longitudinal data collection and a standardized continuous measurement system are fundamental to the evaluation and refinement of complex multicomponent interventions. To be useful to T3-T5 translational research efforts in neighborhoods and communities, such a system would include assessments of the reach, implementation, effects on immediate outcomes, and effects of the comprehensive intervention package on more distal health outcomes. KeywordsHealth, Well-being, Evaluation design, Complex intervention, Translational research Recent research suggests that significant improvements in the life trajectories of at-risk young people are now possible [1]. Unfortunately, actual well-being lags far behind, especially in areas of concentrated poverty, where the risk of young people developing multiple behavioral and health problems is much higher than other neighborhoods [2]. Major advances in population health will not occur unless we translate existing knowledge into effective multicomponent strategies, implement them in at-risk and underserved communities, and develop rigorous evaluation methods to ensure continual improvement. The success of translational research methods to bring about change in entire communities demands the evolution of methods that are most appropriate for the task at hand. In this paper, we present approaches to translational research stages T3-T5 [3] that advance the adoption, implementation, and maintenance of effective and replicable multicomponent interventions. We highlight experimental methods that can sharpen our understanding of the impact of complex interventions as well as each component program, policy, or practice that is implemented to support community development. These experimental methods will enable continuous quality improvement to ensure that neighborhood and community change efforts become increasingly effective and sustained over time. The methods are appropriate and applicable for research stages T3-T5 nationally and globally.Most health outcomes have Bmultiple interacting influences crossing socio-ecological levels^and require complex, multi-level strategies to foster health and well-being [4] To illustrate the need for ...

How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling

“…Our work focuses on the middle link. Recent work around sustainable interventions due to, e.g., Komro et al (2016) and Chambers et al (2013), highlight the import of the third link and work in the area of the economics of charity (see, e.g., List, 2011b) focus on the first link. While a new economic model for these other areas is beyond the scope of this study, we do have thoughts on treatment measurement of a program once it is actually scaled.…”

“…FollowingAl-Ubaydli et al (2019), we view the chain as having three major links: i) funding basic research (seeList, 2011b), ii) providing the knowledge creation market with the optimal incentives for researchers to design, implement, and report scientific results, and iii) developing a system whereby policymakers have the appropriate incentives to adopt effective policies, and once adopted develop strategies to implement those policies with rigorous evaluation methods to ensure continual improvement (see, e.g.,Komro et al, 2016;Chambers et al, 2013).…”

The full translational spectrum of prevention science: facilitating the transfer of knowledge to practices and policies that prevent behavioral health problems

“…Remedies include consideration of the foundational findings and delineating transdisciplinary applications of system science and innovative research techniques in the implementation and evaluation sciences. Also important is to enhance communication between researchers and communities (e.g., practitioners and policy-makers) necessary for eventual acceptance and rigorous adoption [4,[9][10][11][12].…”

25 Years of Complex Intervention Trials

“…As we suggest in our translation research paper (Komro et al, 2016), it is unlikely that the first attempt to implement a complex package of interventions will be totally successful; adjustments will be necessary. Therefore, a stepped-wedge or time-series design with routine and frequent assessments of implementation and intermediate outcomes could simultaneously address the needs for a rigorous evaluation and for a practical way of implementing, refining, and improving the intervention.…”

“…Therefore, a stepped-wedge or time-series design with routine and frequent assessments of implementation and intermediate outcomes could simultaneously address the needs for a rigorous evaluation and for a practical way of implementing, refining, and improving the intervention. In this way, continuous evaluation and refinement of the intervention (including implementation strategies and intervention components) can occur within one community and lessons learned applied to implementation in subsequent communities (Komro et al, 2016). As the complexity and higher order level of response continues to develop, so too do the research design options that need to be considered for rigorous evaluation in real-world settings.…”