Optimally solving Markov decision processes with total expected discounted reward function: Linear programming revisited

Alagöz, Oğuzhan; Ayvaci, Mehmet; Linderoth, Jeff

doi:10.1016/j.cie.2015.05.031

Cited by 11 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent advances in patient admission modeling have also been made by Alagoz et al. (2015), Yang et al. (2015), and Landa et al.…”

Section: Literature Reviewmentioning

confidence: 97%

“…Barz and Rajaram (2015) studied a patient admission problem in a hospital with multiple resource constraints and formulated the control process as an MDP to maximize the expected contribution net of overbooking costs. Recent advances in patient admission modeling have also been made by Alagoz et al (2015), Yang et al (2015), and Landa et al (2018b). For one of the approaching methods to minimize the access to admission control, readers may refer to bed leveling (Beliën and Demeulemeester, 2008;Yip et al, 2018;Aringhieri et al, 2021).…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Optimal admission control under premature discharge decisions for operational effectiveness

Yang

Jiang

Tang

2022

Int Trans Operational Res

View full text Add to dashboard Cite

The shortage of inpatient beds causes surgery cancellations, treatment delays, and inappropriate ward arrangements. That is why studying how to make admission decisions on the wards of crowded units under premature discharge decisions is essential. In this study, we set up an infinite-horizon total discounted cost Markov decision process model to minimize the total expected cost to the hospital. By analyzing the structural properties of the optimal value function, we find that there is an optimal admission control policy of the threshold type for low-acuity emergency patients (type-2). Numerical experiments are done to examine how optimal dynamic policy depends on key system parameters. Furthermore, we evaluate the performance of the optimal dynamic policy with a benchmark policy without premature discharge decisions. The results indicate that the optimal admission control policy helps hospitals in maintaining low costs and enables patients with high-acuity diseases to enter primary wards more quickly.

show abstract

“…Recent advances in patient admission modeling have also been made by Alagoz et al. (2015), Yang et al. (2015), and Landa et al.…”

Section: Literature Reviewmentioning

confidence: 97%

Section: Literature Reviewmentioning

confidence: 99%

Optimal admission control under premature discharge decisions for operational effectiveness

Yang

Jiang

Tang

2022

Int Trans Operational Res

View full text Add to dashboard Cite

show abstract

“…It describes the process of transformation of the system's current state to another state through actions. Recent researches on the applications of MDP and their integration with optimization methods are reviewed in Ahiska et al (2013); Salari and Makis (2017); Ahluwalia et al (2021); Alagoz et al (2015); Steimle et al (2021). Most applications concern transport problems (Yu et al, 2019;Kamrani et al, 2020;Li et al, 2021;Qiu et al, 2022), and these studies show that mathematical programming methods (e.g., LP, MILP) perform efficiently to solve MDP problems case-dependently (Alagoz et al, 2015).…”

Section: Literature Reviewmentioning

confidence: 99%

“…Recent researches on the applications of MDP and their integration with optimization methods are reviewed in Ahiska et al (2013); Salari and Makis (2017); Ahluwalia et al (2021); Alagoz et al (2015); Steimle et al (2021). Most applications concern transport problems (Yu et al, 2019;Kamrani et al, 2020;Li et al, 2021;Qiu et al, 2022), and these studies show that mathematical programming methods (e.g., LP, MILP) perform efficiently to solve MDP problems case-dependently (Alagoz et al, 2015). To overcome the higher complexity of large size instances using mathematical programming, exact optimization approaches such as decomposition (Steimle et al, 2021) and branch-and-bound (Ahluwalia et al, 2021) algorithms can perform well to tackle MDPs.…”

Section: Literature Reviewmentioning

confidence: 99%

Markov decision process for multi-manned mixed-model assembly lines with walking workers

Hasemi-Petroodi

Thevenin

Kovalev

et al. 2023

International Journal of Production Economics

View full text Add to dashboard Cite

“…Markov decision processes (MDPs), referred to as stochastic control problems, are models for sequential decision making when outcomes are uncertain. MDPs have been applied to many areas, including finance, logistics, manufacturing, and health-care [4]. Also, numerous applications and research studies have proven the MDP to be an effective technique for solving optimization problems [5].…”

Section: Introductionmentioning

confidence: 99%

An Optimal Dynamic Admission Control Policy and Upper Bound Analysis in Wireless Sensor Networks

Zhang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Data admission control is an important issue in wireless sensor networks (WSNs) because of the limited transmission coverage area and the limited battery capability of each sensor node. To achieve the optimum benefit of the sensor network, a reward when a data package arriving at a sensor is accepted (not rejected) for transmission is considered, but a holding cost per unit time for the accepted data package waiting in the sensor for transmission is also incurred. For the sensor with sleep and active phases, a dynamic data admission control model is developed in this paper. By constructing a suitable Markov decision process (MDP), we verify that the optimal admission control policy of when to admit or reject an arriving data packet is a control limit policy, in order to achieve the maximum expected a discounted reward in the sleep and active phase, respectively. Furthermore, we provide a formula for how to calculate the upper bound of these threshold values. For the identified model with the optimal expected discount reward, the energy consumption of the sensors in active and sleep phases, as well as the energy consumption switching from active to sleep per unit time and vice versa is investigated. Extensive simulation is implemented. The results show that the problem is effectively solved by an optimal scheme with high energy efficiency. The results of this paper can be applied to designing optimal sensor nodes in wireless sensor networks.INDEX TERMS Admission control, Markov decision process, control limit policy, upper bound, energy consumption.

show abstract

Optimally solving Markov decision processes with total expected discounted reward function: Linear programming revisited

Cited by 11 publications

References 41 publications

Optimal admission control under premature discharge decisions for operational effectiveness

Optimal admission control under premature discharge decisions for operational effectiveness

Markov decision process for multi-manned mixed-model assembly lines with walking workers

An Optimal Dynamic Admission Control Policy and Upper Bound Analysis in Wireless Sensor Networks

Contact Info

Product

Resources

About