Personalized vital signs control based on continuous action-space reinforcement learning with supervised experience

WD3QNE: A value-based deep reinforcement learning model with human expertise in optimal treatment of sepsis

“…Because only the patient's survival is concerned, the reward is observed after a long sequence of decisions. We also apply intermediate rewards and final reward in the form of SOFA change and survival after 90 days respectively 25 . SOFA represents the evidence of organ dysfunction and has been recommended by experts as a screening tool for sepsis 34 .…”

“…Off-policy evaluation. In experiments, we use the intermediate reward parameter 𝛽 𝑠 = 0.6 and the terminal reward parameter 𝛽 𝑇 = 24, following the setting in existing works 25 . Specifically, the terminal reward is 24 if the patient survives, otherwise -24.…”

“…Then, we calculate the average survival rate based on the return value. The survival formula 25 is shown below:…”

“…Nonetheless, a DRL agent only interacts with the environment to seek the optimal actions with high reward, regardless of the potential risks. It has been noted that certain actions induced by AI could cause high risk and lead to contentious medical solutions 25 , which has significantly stymied the broad adoption of AI in healthcare management. On the other front, human experts maintain an edge over AI in abstract reasoning under ambiguous conditions.…”

“…Remarkably, AI or data-driven models suffer from biases in data and model building, and consequently may provoke treatment solutions that are against the principle of clinic practices. To this end, hybrid systems of SL and RL that capitalize on the availability of large-scale EMR have been proposed, which are capable of providing reliable medical recommendations 24,25 . Nevertheless, the usage of SL not only increases the computational complexity but also limits the self-adaptiveness of the RL decision in long-term reward 26 .…”