Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health

Yao, Bing; Chen, Yun; Yang, Hui

doi:10.1109/tase.2021.3052483

“…The prevalence of AF plagues over 2.7 million people in the United States, and this number is estimated to rise to 12.1 million in 2030, as the population ages ( Colilla et al, 2013 ). In healthcare practice, the electrocardiogram (ECG) is a cost-effective and noninvasive medical approach to record the electrical signals on the body surface as a reflection of cardiac health conditions ( Yao and Yang, 2016 ; Yao and Yang, 2020 ; Yao et al, 2021 ; Xie and Yao, 2023 ).…”

Section: Introductionmentioning

confidence: 99%

Automated identification of atrial fibrillation from single-lead ECGs using multi-branching ResNet

Xie,

Stavrakis,

Yao

2024

Front. Physiol.

Self Cite

2

0

View full text Add to dashboard Cite

Introduction: Atrial fibrillation (AF) is the most common cardiac arrhythmia, which is clinically identified with irregular and rapid heartbeat rhythm. AF puts a patient at risk of forming blood clots, which can eventually lead to heart failure, stroke, or even sudden death. Electrocardiography (ECG), which involves acquiring bioelectrical signals from the body surface to reflect heart activity, is a standard procedure for detecting AF. However, the occurrence of AF is often intermittent, costing a significant amount of time and effort from medical doctors to identify AF episodes. Moreover, human error is inevitable, as even experienced medical professionals can overlook or misinterpret subtle signs of AF. As such, it is of critical importance to develop an advanced analytical model that can automatically interpret ECG signals and provide decision support for AF diagnostics.Methods: In this paper, we propose an innovative deep-learning method for automated AF identification using single-lead ECGs. We first extract time-frequency features from ECG signals using continuous wavelet transform (CWT). Second, the convolutional neural networks enhanced with residual learning (ReNet) are employed as the functional approximator to interpret the time-frequency features extracted by CWT. Third, we propose to incorporate a multi-branching structure into the ResNet to address the issue of class imbalance, where normal ECGs significantly outnumber instances of AF in ECG datasets.Results and Discussion: We evaluate the proposed Multi-branching Resnet with CWT (CWT-MB-Resnet) with two ECG datasets, i.e., PhysioNet/CinC challenge 2017 and ECGs obtained from the University of Oklahoma Health Sciences Center (OUHSC). The proposed CWT-MB-Resnet demonstrates robust prediction performance, achieving an F1 score of 0.8865 for the PhysioNet dataset and 0.7369 for the OUHSC dataset. The experimental results signify the model’s superior capability in balancing precision and recall, which is a desired attribute for ensuring reliable medical diagnoses.

show abstract

“…In healthcare, constrained MDPs have been used to assess the effects of budgetary restrictions in biopsy decision making (Ayvaci et al, 2012), incorporating equity considerations in emergency ambulance deployment (McLay and Mayorga, 2013a), liver cancer screening with budget constraints (Chen et al, 2018) and optimal sensing of cardiac events (Yao et al, 2021). Constrained MDPs have been used in other domains as well including operator staffing for call centers (Bhandari et al, 2008) and identifying vertical handoff decisions in wireless networks (Sun et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

Novel Solution Methods for CPOMDPs With Applications to Cancer Screening

Kavaklioglu

2024

Preprint

0

View full text Add to dashboard Cite

<p>Partially Observable Markov Decision Processes (POMDPs) is a modeling framework that allow decision makers to make decisions in uncertain environments. This framework is particularly suited for sequential decision making problems for which the uncertainties are revealed in stages. Constrained POMDPs (CPOMDPs) extend this framework by also accounting for various restrictions in the environment. This PhD thesis focuses on POMDPs, CPOMDPs and its extensions, proposes novel solution methods and studies applications of POMDPs in disease screening. First, we focus on developing efficient versions of the well-known algorithms for POMDPs. POMDPs are notoriously difficult to solve optimally, and exact solution algorithms typically fail to solve the instances with more than a few core states. In this regard, we consider scalable algorithms solve the POMDP problems. Specifically, we propose distributed and parallel versions of the exact solution methods such as Monahan's algorithm and the incremental pruning algorithm as well as the heuristics such as Lovejoy's upper bound and lower bound methods. Next, we study the CPOMDPs and propose linear programming-based solution methods as a flexible approach for those. We perform a detailed numerical study with five different problem instances to showcase the viability of the proposed solution approach for CPOMDPs. Lastly, we propose multi-objective CPOMDP models for the breast cancer screening problem. These CPOMDP models extend the previously proposed POMDP models for breast cancer screening by considering different screening modalities and simultaneously optimizing for expected total quality-adjusted life years maximization and life-time risk minimization objectives.</p>

show abstract

“…In healthcare, constrained MDPs have been used to assess the effects of budgetary restrictions in biopsy decision making (Ayvaci et al, 2012), incorporating equity considerations in emergency ambulance deployment (McLay and Mayorga, 2013a), liver cancer screening with budget constraints (Chen et al, 2018) and optimal sensing of cardiac events (Yao et al, 2021). Constrained MDPs have been used in other domains as well including operator staffing for call centers (Bhandari et al, 2008) and identifying vertical handoff decisions in wireless networks (Sun et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

Novel Solution Methods for CPOMDPs With Applications to Cancer Screening

Kavaklioglu

2024

Preprint

0

View full text Add to dashboard Cite

<p>Partially Observable Markov Decision Processes (POMDPs) is a modeling framework that allow decision makers to make decisions in uncertain environments. This framework is particularly suited for sequential decision making problems for which the uncertainties are revealed in stages. Constrained POMDPs (CPOMDPs) extend this framework by also accounting for various restrictions in the environment. This PhD thesis focuses on POMDPs, CPOMDPs and its extensions, proposes novel solution methods and studies applications of POMDPs in disease screening. First, we focus on developing efficient versions of the well-known algorithms for POMDPs. POMDPs are notoriously difficult to solve optimally, and exact solution algorithms typically fail to solve the instances with more than a few core states. In this regard, we consider scalable algorithms solve the POMDP problems. Specifically, we propose distributed and parallel versions of the exact solution methods such as Monahan's algorithm and the incremental pruning algorithm as well as the heuristics such as Lovejoy's upper bound and lower bound methods. Next, we study the CPOMDPs and propose linear programming-based solution methods as a flexible approach for those. We perform a detailed numerical study with five different problem instances to showcase the viability of the proposed solution approach for CPOMDPs. Lastly, we propose multi-objective CPOMDP models for the breast cancer screening problem. These CPOMDP models extend the previously proposed POMDP models for breast cancer screening by considering different screening modalities and simultaneously optimizing for expected total quality-adjusted life years maximization and life-time risk minimization objectives.</p>

show abstract

Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health

Cited by 10 publications

References 69 publications

Automated identification of atrial fibrillation from single-lead ECGs using multi-branching ResNet

Automated identification of atrial fibrillation from single-lead ECGs using multi-branching ResNet

Novel Solution Methods for CPOMDPs With Applications to Cancer Screening

Novel Solution Methods for CPOMDPs With Applications to Cancer Screening

Contact Info

Product

Resources

About