pH-RL: A Personalization Architecture to Bring Reinforcement Learning to Health Practice

Abstract

While reinforcement learning (RL) has proven to be the approach of choice for tackling many complex problems, it remains challenging to develop and deploy RL agents in real-life scenarios successfully. This paper presents pH-RL (personalization in e-Health with RL), a general RL architecture for personalization to bring RL to health practice. pH-RL allows for various levels of personalization in health applications and allows for online and batch learning. Furthermore, we provide a general-purpose implementation framework that can be integrated with various healthcare applications. We describe a step-by-step guideline for the successful deployment of RL policies in a mobile application. We implemented our open-source RL architecture and integrated it with the MoodBuster mobile application for mental health to provide messages to increase daily adherence to the online therapeutic modules. We then performed a comprehensive study with human participants over a sustained period. Our experimental results show that the developed policies learn to select appropriate actions consistently using only a few days' worth of data. Furthermore, we empirically demonstrate the stability of the learned policies during the study.

Personalized Reinforcement Learning with a Budget of Policies

“…For an in-depth review, see (den Hengst et al 2020). A key challenge is personalizing RL policies in real-world applications, especially in healthcare (Hassouni et al 2018;Zhu et al 2018;Grua and Hoogendoorn 2018;El Hassouni et al 2019;el Hassouni et al 2022). While offering a single policy to all users can be suboptimal, training a policy per user can be an inefficient use of collected samples.…”