Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning

Gönül, Suat; Namlı, Tuncay; Baskaya, Mert; Sınacı, Ali Anıl; Çoşar, Ahmet; Toroslu, İsmail Hakkı

doi:10.1007/978-3-319-92058-0_32

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also presented the core algorithm that is used to optimize the intervention delivery policy in, 48 where we aimed to break the sedentary behaviors of office workers during working hours. In that study, we obtained better results for a machine-learning-based personalization mechanism compared to results obtained for interventions delivered according to a fixed schedule.…”

Section: Discussionmentioning

confidence: 99%

An expandable approach for design and personalization of digital, just-in-time adaptive interventions

Gönül

Namlı

Huisman

et al. 2018

Journal of the American Medical Informatics Association

Self Cite

View full text Add to dashboard Cite

ObjectiveWe aim to deliver a framework with 2 main objectives: 1) facilitating the design of theory-driven, adaptive, digital interventions addressing chronic illnesses or health problems and 2) producing personalized intervention delivery strategies to support self-management by optimizing various intervention components tailored to people’s individual needs, momentary contexts, and psychosocial variables.Materials and MethodsWe propose a template-based digital intervention design mechanism enabling the configuration of evidence-based, just-in-time, adaptive intervention components. The design mechanism incorporates a rule definition language enabling experts to specify triggering conditions for interventions based on momentary and historical contextual/personal data. The framework continuously monitors and processes personal data space and evaluates intervention-triggering conditions. We benefit from reinforcement learning methods to develop personalized intervention delivery strategies with respect to timing, frequency, and type (content) of interventions. To validate the personalization algorithm, we lay out a simulation testbed with 2 personas, differing in their various simulated real-life conditions.ResultsWe evaluate the design mechanism by presenting example intervention definitions based on behavior change taxonomies and clinical guidelines. Furthermore, we provide intervention definitions for a real-world care program targeting diabetes patients. Finally, we validate the personalized delivery mechanism through a set of hypotheses, asserting certain ways of adaptation in the delivery strategy, according to the differences in simulation related to personal preferences, traits, and lifestyle patterns.ConclusionWhile the design mechanism is sufficiently expandable to meet the theoretical and clinical intervention design requirements, the personalization algorithm is capable of adapting intervention delivery strategies for simulated real-life conditions.

show abstract

Section: Discussionmentioning

confidence: 99%

An expandable approach for design and personalization of digital, just-in-time adaptive interventions

Gönül

Namlı

Huisman

et al. 2018

Journal of the American Medical Informatics Association

Self Cite

View full text Add to dashboard Cite

show abstract

“…( 14 , 49 )]. To optimize timing or content of the reminders, machine learning methods have been presented ( 50 , 51 ).…”

Section: Methodsmentioning

confidence: 99%

The Design and Development of a Personalized Leisure Time Physical Activity Application Based on Behavior Change Theories, End-User Perceptions, and Principles From Empirical Data Mining

Sporrel

Boer

Wang

et al. 2021

Front. Public Health

View full text Add to dashboard Cite

Introduction: Many adults do not reach the recommended physical activity (PA) guidelines, which can lead to serious health problems. A promising method to increase PA is the use of smartphone PA applications. However, despite the development and evaluation of multiple PA apps, it remains unclear how to develop and design engaging and effective PA apps. Furthermore, little is known on ways to harness the potential of artificial intelligence for developing personalized apps. In this paper, we describe the design and development of the Playful data-driven Active Urban Living (PAUL): a personalized PA application.Methods: The two-phased development process of the PAUL apps rests on principles from the behavior change model; the Integrate, Design, Assess, and Share (IDEAS) framework; and the behavioral intervention technology (BIT) model. During the first phase, we explored whether location-specific information on performing PA in the built environment is an enhancement to a PA app. During the second phase, the other modules of the app were developed. To this end, we first build the theoretical foundation for the PAUL intervention by performing a literature study. Next, a focus group study was performed to translate the theoretical foundations and the needs and wishes in a set of user requirements. Since the participants indicated the need for reminders at a for-them-relevant moment, we developed a self-learning module for the timing of the reminders. To initialize this module, a data-mining study was performed with historical running data to determine good situations for running.Results: The results of these studies informed the design of a personalized mobile health (mHealth) application for running, walking, and performing strength exercises. The app is implemented as a set of modules based on the persuasive strategies “monitoring of behavior,” “feedback,” “goal setting,” “reminders,” “rewards,” and “providing instruction.” An architecture was set up consisting of a smartphone app for the user, a back-end server for storage and adaptivity, and a research portal to provide access to the research team.Conclusions: The interdisciplinary research encompassing psychology, human movement sciences, computer science, and artificial intelligence has led to a theoretically and empirically driven leisure time PA application. In the current phase, the feasibility of the PAUL app is being assessed.

show abstract

“…The decision rule uses tailoring variables to identify the current state of vulnerability and specifies when it is appropriate to offer intervention [ 21 ]. Owing to substantial individual variability in what tailoring variables and at which thresholds indicate a heightened state of lapse risk, a machine learning algorithm informs the decision rule in this JITAI for lapses [ 6 , 24 - 26 ]. In formative work to develop this JITAI, a supervised machine learning approach was used to train an algorithm using previously collected data on tailoring variables and dietary lapses.…”

Section: Methodsmentioning

confidence: 99%

Optimizing a Just-in-Time Adaptive Intervention to Improve Dietary Adherence in Behavioral Obesity Treatment: Protocol for a Microrandomized Trial

Goldstein¹,

Zhang²,

Klasnja³

et al. 2021

JMIR Res Protoc

View full text Add to dashboard Cite

Background Behavioral obesity treatment (BOT) is a gold standard approach to weight loss and reduces the risk of cardiovascular disease. However, frequent lapses from the recommended diet stymie weight loss and prevent individuals from actualizing the health benefits of BOT. There is a need for innovative treatment solutions to improve adherence to the prescribed diet in BOT. Objective The aim of this study is to optimize a smartphone-based just-in-time adaptive intervention (JITAI) that uses daily surveys to assess triggers for dietary lapses and deliver interventions when the risk of lapse is high. A microrandomized trial design will evaluate the efficacy of any interventions (ie, theory-driven or a generic alert to risk) on the proximal outcome of lapses during BOT, compare the effects of theory-driven interventions with generic risk alerts on the proximal outcome of lapse, and examine contextual moderators of interventions. Methods Adults with overweight or obesity and cardiovascular disease risk (n=159) will participate in a 6-month web-based BOT while using the JITAI to prevent dietary lapses. Each time the JITAI detects elevated lapse risk, the participant will be randomized to no intervention, a generic risk alert, or 1 of 4 theory-driven interventions (ie, enhanced education, building self-efficacy, fostering motivation, and improving self-regulation). The primary outcome will be the occurrence of lapse in the 2.5 hours following randomization. Contextual moderators of intervention efficacy will also be explored (eg, location and time of day). The data will inform an optimized JITAI that selects the theory-driven approach most likely to prevent lapses in a given moment. Results The recruitment for the microrandomized trial began on April 19, 2021, and is ongoing. Conclusions This study will optimize a JITAI for dietary lapses so that it empirically tailors the provision of evidence-based intervention to the individual and context. The finalized JITAI will be evaluated for efficacy in a future randomized controlled trial of distal health outcomes (eg, weight loss). Trial Registration ClinicalTrials.gov NCT04784585; http://clinicaltrials.gov/ct2/show/NCT04784585 International Registered Report Identifier (IRRID) DERR1-10.2196/33568

show abstract

Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning

Cited by 7 publications

References 11 publications

An expandable approach for design and personalization of digital, just-in-time adaptive interventions

An expandable approach for design and personalization of digital, just-in-time adaptive interventions

The Design and Development of a Personalized Leisure Time Physical Activity Application Based on Behavior Change Theories, End-User Perceptions, and Principles From Empirical Data Mining

Optimizing a Just-in-Time Adaptive Intervention to Improve Dietary Adherence in Behavioral Obesity Treatment: Protocol for a Microrandomized Trial

Contact Info

Product

Resources

About