Personalization of Health Interventions Using Cluster-Based Reinforcement Learning

Hassouni, Ali el; Hoogendoorn, Mark; Otterlo, Martijn van; Barbaro, Eduardo

doi:10.1007/978-3-030-03098-8_31

Cited by 21 publications

(37 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If this is the case, then trajectories D j from some similar environment M j = M i may prove useful in estimating E π i [G]. One such an approach is based on clustering [54,75,124,191]. Formally, it requires q n groups G ∈ {g 1 , .…”

Section: Den Hengst Et Al / Reinforcement Learning For Personalizatimentioning

confidence: 99%

Reinforcement learning for personalization: A systematic literature review

Hengst

Grua

Hassouni

et al. 2020

Self Cite

View full text Add to dashboard Cite

The major application areas of reinforcement learning (RL) have traditionally been game playing and continuous control. In recent years, however, RL has been increasingly applied in systems that interact with humans. RL can personalize digital systems to make them more relevant to individual users. Challenges in personalization settings may be different from challenges found in traditional application areas of RL. An overview of work that uses RL for personalization, however, is lacking. In this work, we introduce a framework of personalization settings and use it in a systematic literature review. Besides setting, we review solutions and evaluation strategies. Results show that RL has been increasingly applied to personalization problems and realistic evaluations have become more prevalent. RL has become sufficiently robust to apply in contexts that involve humans and the field as a whole is growing. However, it seems not to be maturing: the ratios of studies that include a comparison or a realistic evaluation are not showing upward trends and the vast majority of algorithms are used only once. This review can be used to find related work across domains, provides insights into the state of the field and identifies opportunities for future work.

show abstract

Section: Den Hengst Et Al / Reinforcement Learning For Personalizatimentioning

confidence: 99%

Reinforcement learning for personalization: A systematic literature review

Hengst

Grua

Hassouni

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…We think this phenomenon will be taken into account in the future study. One possible solution is to give a different initialized strategy to the different user clusters based on their frequency preference [ 34 ].…”

Section: Discussionmentioning

confidence: 99%

Reinforcement Learning to Send Reminders at Right Moments in Smartphone Exercise Application: A Feasibility Study

Wang

Sporrel

Hoof

et al. 2021

IJERPH

View full text Add to dashboard Cite

Just-in-time adaptive intervention (JITAI) has gained attention recently and previous studies have indicated that it is an effective strategy in the field of mobile healthcare intervention. Identifying the right moment for the intervention is a crucial component. In this paper the reinforcement learning (RL) technique has been used in a smartphone exercise application to promote physical activity. This RL model determines the ‘right’ time to deliver a restricted number of notifications adaptively, with respect to users’ temporary context information (i.e., time and calendar). A four-week trial study was conducted to examine the feasibility of our model with real target users. JITAI reminders were sent by the RL model in the fourth week of the intervention, while the participants could only access the app’s other functionalities during the first 3 weeks. Eleven target users registered for this study, and the data from 7 participants using the application for 4 weeks and receiving the intervening reminders were analyzed. Not only were the reaction behaviors of users after receiving the reminders analyzed from the application data, but the user experience with the reminders was also explored in a questionnaire and exit interviews. The results show that 83.3% reminders sent at adaptive moments were able to elicit user reaction within 50 min, and 66.7% of physical activities in the intervention week were performed within 5 h of the delivery of a reminder. Our findings indicated the usability of the RL model, while the timing of the moments to deliver reminders can be further improved based on lessons learned.

show abstract

“…Equally important is to focus more on utilizing model checking tools [2,16]. Application-wise, there are plenty of opportunities to utilize the methods in this paper, for example to constrain RL dialogue agents, in medical domains with logically represented medical guidance and regulations, or to implement coaching strategies in RL coaching agents [17].…”

Section: Discussionmentioning

confidence: 99%

Regular Decision Processes for Grid Worlds

Lenaers¹,

Otterlo²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and reward structures, extensions are needed. To that end, in recent years interest has grown into combinations of reinforcement learning and temporal logic, that is, combinations of flexible behavior learning methods with robust verification and guarantees. In this paper we describe an experimental investigation of the recently introduced regular decision processes that support both non-Markovian reward functions as well as transition functions. In particular, we provide a tool chain for regular decision processes, algorithmic extensions relating to online, incremental learning, an empirical evaluation of model-free and model-based solution algorithms, and applications in regular, but non-Markovian, grid worlds.

show abstract

Personalization of Health Interventions Using Cluster-Based Reinforcement Learning

Cited by 21 publications

References 16 publications

Reinforcement learning for personalization: A systematic literature review

Reinforcement learning for personalization: A systematic literature review

Reinforcement Learning to Send Reminders at Right Moments in Smartphone Exercise Application: A Feasibility Study

Regular Decision Processes for Grid Worlds

Contact Info

Product

Resources

About