Agent-based modeling for implementation research: An application to tobacco smoking cessation for persons with serious mental illness

Abstract

Background: Implementation researchers have sought ways to use simulations to support the core components of implementation, which typically include assessing the need for change, designing implementation strategies, executing the strategies, and evaluating outcomes. The goal of this article is to explain how agent-based modeling could fulfill this role. Methods: We describe agent-based modeling with respect to other simulation methods that have been used in implementation science, using non-technical language that is broadly accessible. We then provide a stepwise procedure for developing agent-based models of implementation processes. We use, as a case study to illustrate the procedure, the implementation of evidence-based smoking cessation practices for persons with serious mental illness (SMI) in community mental health clinics. Results: For our case study, we present descriptions of the motivating research questions, specific models used to answer these questions, and a summary of the insights that can be obtained from the models. In the first example, we use a simple form of agent-based modeling to simulate the observed smoking behaviors of persons with SMI in a recently completed trial (IDEAL, Comprehensive Cardiovascular Risk Reduction Trial in Persons with SMI). In the second example, we illustrate how a more complex agent-based approach that includes interactions between patients, providers, and site administrators can be used to provide guidance for an implementation intervention that includes training and organizational strategies. This example is based in part on an ongoing project focused on scaling up evidence-based tobacco smoking cessation practices in community mental health clinics in Maryland. Conclusion: In this article, we explain how agent-based models can be used to address implementation science research questions and provide a procedure for setting up simulation models. Through our examples, we show how what-if scenarios can be examined in the implementation process, which are particularly useful in implementation frameworks with adaptive components. Plain Language Summary: The goal of this paper is to explain how agent-based modeling could be used as a supplementary tool to support the components of complex implementation processes. Such models have not yet been widely used in implementation science, partly because they are not straightforward to develop. To promote the use of agent-based modeling we provide a stepwise procedure using non-technical language and emphasizing the relationships between the model and implementation processes. We used two detailed examples to demonstrate our proposed approach. In the first example, we simulate the observed smoking behaviors of persons with serious mental illness in a recently completed trial (IDEAL, Comprehensive Cardiovascular Risk Reduction Trial in Persons with Serious Mental Illness). In the second example, we illustrate how agent-based models that include interactions between patients, providers and site administrators can be used to provide guidance for an implementation intervention that includes training and organizational strategies. This example is based in part on an ongoing project focused on scaling up evidence-based tobacco smoking cessation practices in community mental health clinics in Maryland. For this example, we show how the visual user interface of an agent-based model can be in the form of a dashboard with levers for simulating what-if scenarios that can be used to guide implementation decisions. In summary, this paper shows how agent-based models can provide insights into the processes in complex interventions, and guide implementation decisions for improving delivery of evidence-based practices in community mental health clinics.

Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science

“…To do this, the modelers used empirical data including nonexperimental causal inference studies quantifying the effects of alcohol taxes on alcohol consumption to calibrate (by comparing agent-based-model estimates to empirical estimates) the model. Data and measurement challenges are a key limitation of these models, which need to be “parameterized” with data from other studies ( Huang et al, 2021 ). Bayesian techniques for calibrating system science models, such as the Approximate Bayesian Computation method, have emerged as efficient tools that integrate simulation with prior information on uncertain model parameters ( Pritchard et al, 1999 ).…”

Using simulation modeling to inform intervention and implementation selection in a rapid stakeholder-engaged hybrid effectiveness-implementation randomized trial

“…While our team began this project prior to the publication of the RCSM framework, our goals and process corresponded to that recommended in RCSM, including determining stakeholders’ needs, refining a simulation model, and utilizing stakeholder input to iterate modeling [ 60 ]. Huang and colleagues also recently describe the use of agent-based and microsimulation models to inform implementation trials [ 59 ]. Our work builds on that analysis by adding networking and incorporating a disease transmission probability, allowing us to examine the spread of an infectious disease in the population.…”

“…A limited existing literature supports the use of mathematical simulation modeling to inform the design of research trials, albeit primarily clinical trials [53][54][55][56][57]. While recent literature has highlighted the potential value of using simulation modeling in the design of implementation trials [58][59][60], there is a lack of data demonstrating the feasibility and actual use of simulation modeling in the design of implementation research trials, especially for the design of hybrid effectiveness-implementation trials. Sheldrick et al recently suggested guidelines to inform the use of simulation modeling in implementation -a process they named rapid-cycle systems modeling (RCSM) [60].…”

“…Our work builds on that analysis by adding networking and incorporating a disease transmission probability, allowing us to examine the spread of an infectious disease in the population. Moreover, Huang et al detail the process of developing a new model, which can take substantial additional time and resources when designing an implementation trial [59]. In contrast, our study benefited from expansion of an existing, validated simulation model, which allowed both for accuracy and rapidity of iterations in the context of the fast-moving pandemic [40][41][42][61][62][63].…”

Microsimulation Models on Mental Health: A Critical Review of the Literature

“…After titles and abstracts were reviewed, 36 full texts were assessed. Among these, 17 studies were ultimately included in the final review along with two other articles obtained from additional sources (i.e., Google search), [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40] for a total of 19 studies (see Figure 1). As a result, there was an 8.4% (19/227) of relevant hits.…”