End-to-end Personalization of Digital Health Interventions using Raw Sensor Data with Deep Reinforcement Learning

Abstract

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Reinforcement learning for personalization: A systematic literature review

“…For multi-step decision making problems, methods that can estimate the performance of some policy based on data generated by another policy have been developed [37,90,204]. Secondly, advances in the field of deep learning have wholly or partly removed the need for feature engineering [53]. This may be especially challenging for sequential decision-making problems as different features may be of importance in different states encountered over time.…”

pH-RL: A Personalization Architecture to Bring Reinforcement Learning to Health Practice

“…They show that clustering using traces of states and reward and developing policies based on these clusters leads to improved personalization levels while speeding-up the learning time of the approach. In a later work, [5] demonstrates that this approach leads to improved personalization levels when applied on state representations consisting of raw sensor data obtained from mobile apps. Similarly, k-means clustering and RL were combined to develop policies across similar users for the purpose of learning better policies [24].…”

“…Many practical limitations arise in traditional domains such as healthcare, making these benefits listed above fade away [4,5,9,10]. Such limitations are the inaccessibility to large samples of data, the unavailability of environments to train and evaluate algorithms in, the limitations on the data caused by privacy laws, and safety concerns (e.g., unsafe actions and exploration), explainability, and legal responsibility [4,10].…”

“…In the pH-RL framework, we mitigate this issue and propose to group users with similar behavior using clustering techniques [9,5]. Clustering algorithms such as Kmedoids and K-means have been shown to perform well on similar problems in e-Health [9,5,7].…”

“…In the pH-RL framework, we mitigate this issue and propose to group users with similar behavior using clustering techniques [9,5]. Clustering algorithms such as Kmedoids and K-means have been shown to perform well on similar problems in e-Health [9,5,7]. We compare the behavioral traces of users consisting of states and rewards, and we use the Dynamic Time Warping (DTW) algorithm to calculate the distance between two users.…”