Model Exploration of an Information-based Healthcare Intervention using Parallelization and Active Learning

Abstract

This paper describes the application of a large-scale active learning method to characterize the parameter space of a computational agent-based model developed to investigate the impact of CommunityRx, a clinical information-based health intervention that provides patients with personalized information about local community resources to meet basic and self-care needs. The di usion of information about community resources and their use is modeled via networked interactions and their subsequent e ect on agents' use of community resources across an urban population. A random forest model is iteratively fitted to model evaluations to characterize the model parameter space with respect to observed empirical data. We demonstrate the feasibility of using high-performance computing and active learning model exploration techniques to characterize large parameter spaces; by partitioning the parameter space into potentially viable and non-viable regions, we rule out regions of space where simulation output is implausible to observed empirical data. We argue that such methods are necessary to enable model exploration in complex computational models that incorporate increasingly available micro-level behavior data. We provide public access to the model and high-performance computing experimentation code.

Building and experimenting with an agent-based model to study the population-level impact of CommunityRx, a clinic-based community resource referral intervention

“…Combined with prior validation work, these results help to establish the model's readiness for hypothesis-testing experimentation. [38,39] To illustrate the potential of the ABM as a complement to clinical methods, we also conducted an in silico experiment to examine how specific resource information disseminates from an individual exposed to the CRx intervention in a clinical setting ("clinical dose") to other individuals in the population through social dynamics ("social dose"). Through 45 experimental runs simulating mixing behaviors in a geographic population, this study finds that personalized community resource information delivered to a patient during a healthcare visit spreads to others in the community.…”

“…[37] We have previously described in the systems science literature the technical design details of the ABM used in this study as well as the associated model validation. [38,39] Informed by and extending this work, the goals of this study are to: (1) describe the interdisciplinary method of model building and data sources used to develop an ABM to study an information-based health intervention, and (2) describe an in silico experiment to illustrate how an ABM can generate insights to complement a clinical trial of an information-based intervention. First, using the development of the CRx ABM as a case study, we describe interdisciplinary model building methods that could generalize to other information-based healthcare interventions.…”

“…The interdisciplinary process for building the CRx ABM involved several steps, informed by extant literature [21] and illustrated in Fig 3 . Investigators, including physician scientists, epidemiologists, biostatisticians and systems scientists from the University of Chicago Biological Sciences Division and the Decision and Infrastructure Sciences Division and Consortium for Advanced Science and Engineering at Argonne National Laboratory, met regularly over the course of several years to: (1) come to a shared understanding about and vocabulary to enable interdisciplinary study of the CRx intervention, (2) identify and integrate public use, clinical trial and other data sources with the ABM drawing on complementary expertise (e.g. the systems scientsts were experienced with time use data and the others were experienced with public health and trial data), (3) fill data gaps by desiging and implementing new data collection strategies drawing on qualitative research expertise of both groups and survey research skills of the biomedical scientists, (4) describe and validate the model, [39] and (5) design and implement in silico experimentation (a focus of this paper).…”

“…The parameter values for these effects were critical to model calibration and was previously described in detail. [39] 3.5.4 Agent decision-making. We then developed processes for agents' decisions about using a resource (recall, defined as a specific service at a specific place).…”

“…Sensitivity analyses performed on selected parameters have been previously described. [39] The agent's decision to use a resource was modeled as a binary choice A/B Decision Model. Each activity was characterized by a decision type-A/B decision or not.…”