Planning treatment of ischemic heart disease with partially observable Markov decision processes

Hauskrecht, Miloš; Fraser, Hamish

doi:10.1016/s0933-3657(99)00042-1

Cited by 141 publications

(102 citation statements)

References 13 publications

Supporting

Mentioning

100

Contrasting

Unclassified

Order By: Relevance

“…Hu et al (1993) considered the problem of choosing an appropriate drug infusion plan for the administration of anesthesia. Hauskrecht and Fraser (2000) applied a POMDP formulation to the problem of treating patients with ischemic heart disease. Maillart et al (2008) used a partially observable Markov process to study breast cancer screening policies using mammography; they evaluated age-dependent screening policies and studied the tradeoff between lifetime mortality risk of breast cancer and the expected number of mammograms.…”

Section: Introductionmentioning

confidence: 99%

Optimization of Prostate Biopsy Referral Decisions

Zhang

Denton

Balasubramanian

et al. 2012

M&SOM

View full text Add to dashboard Cite

Prostate cancer is the most common solid tumor in American men and is screened for using prostate-specific antigen (PSA) tests. We report on a non-stationary partially observable Markov decision process (POMDP) for prostate biopsy referral decisions. The core states are the patients' prostate cancer related health states, and PSA test results are the observations. Transition probabilities and rewards are inferred from the Mayo Clinic Radical Prostatectomy Registry (MCRPR) and the medical literature. The objective of our model is to maximize expected quality-adjusted life years (QALYs). We solve the POMDP model to obtain an age and belief (probability of having prostate cancer) dependent optimal biopsy referral policy. We also prove a number of structural properties including the existence of a control-limit type policy for the biopsy referral decision. Our empirical results demonstrate a non-decreasing belief threshold in age, and we provide sufficient conditions under which PSA screening should be discontinued for older patients. Finally, the benefits of screening under the optimal biopsy referral policy are estimated, and sensitivity analysis is used to prioritize the model parameters that would benefit from additional data collection.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimization of Prostate Biopsy Referral Decisions

Zhang

Denton

Balasubramanian

et al. 2012

M&SOM

View full text Add to dashboard Cite

show abstract

“…Our model can also be used for better informed budget allocation and resource planning decisions evaluating the combined cost measure of the dead-alive trade off, quality of life and economic costs [39]. This can be achieved by using our model coupled with a partially observable Markov decision process based on, for example, the patient management model proposed by [39] to select the care pathway which optimizes the combined cost for the integrated care system.…”

Section: Discussionmentioning

confidence: 99%

“…This can be achieved by using our model coupled with a partially observable Markov decision process based on, for example, the patient management model proposed by [39] to select the care pathway which optimizes the combined cost for the integrated care system. This will also facilitate cost effectiveness analysis (CEA) [40][41][42] and cost-utility analysis (CUA) [40,43] of different admission scheduling policies and resource management strategies.…”

Section: Discussionmentioning

confidence: 99%

A non-homogeneous discrete time Markov model for admission scheduling and resource planning in a cost or capacity constrained healthcare system

Garg

McClean

Meenan

et al. 2009

Health Care Manag Sci

View full text Add to dashboard Cite

Healthcare resource planners need to develop policies that ensure optimal allocation of scarce healthcare resources. This goal can be achieved by forecasting daily resource requirements for a given admission policy. If resources are limited, admission should be scheduled according to the resource availability. Such resource availability or demand can change with time. We here model patient flow through the care system as a discrete time Markov chain. In order to have a more realistic representation, a nonhomogeneous model is developed which incorporates timedependent covariates, namely a patient's present age and the present calendar year. The model presented in this paper can be used for admission scheduling, resource requirement forecasting and resource allocation, so as to satisfy the demand or resource constraints or to meet the expansion or contraction plans in a hospital and community based integrated care system. Such a model can be used with both fixed and variable numbers of admissions per day and should prove to be a useful tool for care managers and policy makers who require to make strategic management decisions. We also describe an application of the model to an elderly care system, using a historical dataset from the geriatric department of a London hospital.

show abstract

“…To solve a MOMDP, the main modifications required for SARSOP are the belief update equation (8) and the backup operations on the lower bound (Algorithm 2) and the upper bound (Algorithm 3). These primitive operations do not affect the convergence property of SARSOP, and the new algorithm provides the same theoretical guarantee as the original SARSOP algorithm [14].…”

Section: Convergencementioning

confidence: 99%

Planning under Uncertainty for Robotic Tasks with Mixed Observability

Ong

Png

Hsu

et al. 2010

The International Journal of Robotics Research

182

165

View full text Add to dashboard Cite

Partially observable Markov decision processes (POMDPs) provide a principled, general framework for robot motion planning in uncertain and dynamic environments. They have been applied to various robotic tasks. However, solving POMDPs exactly is computationally intractable. A major challenge is to scale up POMDP algorithms for complex robotic tasks.Robotic systems often have mixed observability: even when a robot's state is not fully observable, some components of the state may still be so. We use a factored model to represent separately the fully and partially observable components of a robot's state and derive a compact lower-dimensional representation of its belief space. This factored representation can be combined with any point-based algorithm to compute approximate POMDP solutions. Experimental results show that on standard test problems, our approach improves the performance of a leading point-based POMDP algorithm by many times.

show abstract

Planning treatment of ischemic heart disease with partially observable Markov decision processes

Cited by 141 publications

References 13 publications

Optimization of Prostate Biopsy Referral Decisions

Optimization of Prostate Biopsy Referral Decisions

A non-homogeneous discrete time Markov model for admission scheduling and resource planning in a cost or capacity constrained healthcare system

Planning under Uncertainty for Robotic Tasks with Mixed Observability

Contact Info

Product

Resources

About