An interpretable health behavioral intervention policy for mobile device users

Abstract

An increasing number of people use mobile devices to monitor their behavior, such as exercise, and record their health status, such as psychological stress. However, these devices rarely provide ongoing support to help users understand how their behavior contributes to changes in their health status. To address this challenge, we aim to develop an interpretable policy for physical activity recommendations that reduce a user’s perceived psychological stress, over a given time horizon. We formulate this problem as a sequential decision-making problem and solve it using a new method that we refer to as threshold Q-learning (TQL). The advantage of the TQL method over traditional Q-learning is that it is “doubly robust” and interpretable. This interpretability is achieved by making model assumptions and incorporating threshold selection into the learning process. Our simulation results indicate that the TQL method performs better than the Q-learning method given model misspecification. Our analyses are performed on data collected from 79 healthy adults over a 7 week period, where the data comprise physical activity patterns collected from mobile devices and self-assessed stress levels of the users. This work serves as a first step toward a computational health coaching solution for mobile device users.

Molecular Simulations of Hydrophobic Gating of Pentameric Ligand Gated Ion Channels: Insights into Water and Ions

“…In combination with implementation of biomolecular simulation codes on exascale computing resources 121 − 124 and growth in the number of ion channel structures, 125 these advances in force fields for channel simulations offer the possibility of, e.g., global comparisons of ion channel conductance behaviors obtained by large scale Comp ePhys surveys with more accurate models of ion/water/channel interactions, thus yielding new insights into the atomic level relationship between structure and physiological function in biological ion channels.…”

Patient Heal Thyself: Reducing Hospital Readmissions with Technology‐Enabled Continuity of Care and Patient Activation

“…This feedback from patient to health care provider and back can occur through face‐to‐face appointments or digital communications (Hu et al. ). Regardless of mode, this feedback enables health care providers to improve support to specific individuals (Street et al.…”

“…Patients provide aggregate feedback into the health care provision system that allows health care providers to learn from the patients and refine their recommendations or support to an individual patient. This feedback from patient to health care provider and back can occur through face-to-face appointments or digital communications (Hu et al 2018). Regardless of mode, this feedback enables health care providers to improve support to specific individuals (Street et al 2009) and focus on challenges specific to a patient.…”

“…Follow on work in patient activation ought to examine fairness issues when PAM predictions are used in medical decision-making. In parallel, there is growing interest in using personalized technology to provide interventions specific to a patient's context to support healthier behaviors (Hu et al 2018). Thus, it ought to be possible to introduce additional constructs that refine predictions of patient activation.…”

Reinforcement Learning Interpretation Methods: A Survey

“…A more formal way to define ML systems is the one proposed by Mitchel [17] "A machine is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E". Reinforcement Learning (RL) is one of the emerging sub-fields of ML that achieved outstanding performance in different domains such as robotics [18], self driving cars [19], health informatics [20], beating humans in complex games such as Go [21], and playing Atari games [22]. It studies the systems that can be represented with an agent which interacts with a dynamic environment through taking different actions in order to maximize its cumulative reward [23].…”