Data-Driven Markov Decision Process Approximations for Personalized Hypertension Treatment Planning

Abstract

Background: Markov decision process (MDP) models are powerful tools. They enable the derivation of optimal treatment policies but may incur long computational times and generate decision rules that are challenging to interpret by physicians. Methods: In an effort to improve usability and interpretability, we examined whether Poisson regression can approximate optimal hypertension treatment policies derived by an MDP for maximizing a patient’s expected discounted quality-adjusted life years. Results: We found that our Poisson approximation to the optimal treatment policy matched the optimal policy in 99% of cases. This high accuracy translates to nearly identical health outcomes for patients. Furthermore, the Poisson approximation results in 104 additional quality-adjusted life years per 1000 patients compared to the Seventh Joint National Committee’s treatment guidelines for hypertension. The comparative health performance of the Poisson approximation was robust to the cardiovascular disease risk calculator used and calculator calibration error. Limitations: Our results are based on Markov chain modeling. Conclusions: Poisson model approximation for blood pressure treatment planning has high fidelity to optimal MDP treatment policies, which can improve usability and enhance transparency of more personalized treatment policies.

Personalizing Second-Line Type 2 Diabetes Treatment Selection: Combining Network Meta-analysis, Individualized Risk, and Patient Preferences for Unified Decision Support

“… CVD, cardiovascular disease; DPP-4-I, dipeptidyl peptidase 4 inhibitor; GLP-1RA, glucagon-like peptide 1 agonist; SGLT-2-I, sodium-glucose linked transporter 2 inhibitor, SU, sulfonylurea; TZD, thiazolidinedione. a. Preference weights: Only macrovascular and microvascular outcomes were considered important, accounting for the disability weights associated with disease outcomes. 42–46 b. Drug-ranking scores were normalized by the best ranking score. A score of 1 refers to the best ranking score, and the scores for other drugs are ranking scores relative the best ranking score.…”

“… a. Preference weights: Only macrovascular and microvascular outcomes were considered important, accounting for the disability weights associated with disease outcomes. 42–46 …”

“…Preference weight of 1 (very important) was assigned to each outcome at a time with the rest of outcomes set to 0 (not important) Preference weights: Only macrovascular and microvascular outcomes were considered important, accounting for the disability weights associated with disease outcomes. [36][37][38][39][40] * Drug ranking scores were normalized by the best ranking score. Score of 1 refers to the best ranking score, and the scores for other drugs are ranking scores relative the best ranking score.…”

“…To account for the relative severity/importance of decision outcomes, normalized disability weights associated with each outcome were incorporated based on prior comprehensive surveys of quality-adjusted disutility (Supplementary Appendix Table 3), separately from patient preference weights. [42][43][44][45][46] For example, if a patient places preference weights of 1 (very important) to avoid both CVD and cost, the normalized weights would adjust the weights so that the ranking score considers avoiding CVD more seriously than avoiding higher cost with higher disability weights placed on CVD; the ranking score becomes less sensitive to avoiding higher cost as compared with avoiding stroke (Supplementary Appendix Table 4). We adjust for both the severity of the condition (disability weights) and how much a patient ''cares'' about the risk of experiencing that condition or not (preference weights, reflecting factors such as risk aversion and discounting), which are conceptually distinct.…”

Dynamic treatment selection and modification for personalised blood pressure therapy using a Markov decision process model: a cost-effectiveness analysis

“…Prior work suggests that personalising optimal treatment policies using an MDP framework could improve patient health outcomes compared with JNC7 treatment guidelines. 11 This prior work focuses on assessing the impact of the MDP-based policies on certain cohorts of patients, not on population health. Our study advances this prior literature by incorporating large-scale meta-analytic and network meta-analytic data sources, providing a full life-course simulation to evaluate long-term impact and using population-representative data to assess the overall national implications of personalised blood pressure treatment selection.…”