Effects of System Parameters on the Optimal Cost and Policy in a Class of Multidimensional Queueing Control Problems

Vercraene, Samuel; Gayon, Jean-Philippe; Karaesmen, Fikri

doi:10.1287/opre.2017.1600

Cited by 9 publications

(3 citation statements)

References 20 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While they did not derive the structure of the optimal policy when the return lead-time is positive. This is owed to the great complexity which could be led by the difficulty analysis of multidimensional MDP [25]. The work by Zerhouni et al [6] differs from ours in that we consider the queueing model with level dependent returns, which lead to our model formulation to be a two-dimensional Markov chain.…”

Section: Literature Reviewmentioning

confidence: 90%

Optimal Production Control of Remanufacturing Systems Based on a Queueing Model With Returns

Chen

2021

IEEE Access

View full text Add to dashboard Cite

This paper analyzes an optimal dynamic production control problem of a remanufacturing system in stochastic environment. The system consists of one inventory for serviceable items and one virtual inventory for the items used by customers which will be either remanufactured to be serviceable items or discarded. Demands will be lost if there is no serviceable items available. At any decision epoch, the decision maker has to decide whether to produce items or not by raw materials, so as to minimize the long run average cost. The optimization problem is constructed as a Markov decision process (MDP). We show that the optimal policy has a multi-threshold structure (switching curve) and it is monotonic in system parameters. Furthermore, based on the structure of the optimal policy, we construct a performance evaluation model for computing efficiently the optimal thresholds. The expression of the average cost is given by a quasi-birthdeath (QBD) process. Finally, we provide some numerical results to show the effectiveness of our approach and the impact of different parameters on the optimal policy and average cost.

show abstract

Section: Literature Reviewmentioning

confidence: 90%

Optimal Production Control of Remanufacturing Systems Based on a Queueing Model With Returns

Chen

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Chou and Abraham [79], e Silva and Gerla [288], Franken and Haverkort [128], Li and Kameda [217], Nadar et al [244] and Vercraene et al [316].…”

Section: (7) Mdps Of Distributed Queueing Networkmentioning

confidence: 99%

An Overview for Markov Decision Processes in Queues and Networks

Fan

et al. 2019

Stochastic Models in Reliability, Network Security and System Safety

View full text Add to dashboard Cite

Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper Provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this paper summarizes several interesting directions in the future research. We hope that this overview can shed light to MDPs in queues and networks, and also to their extensive applications in various practical areas.One main purpose of this paper is to provide an overview for research on MDPs in queues and networks in the last six decades. Also, such a survey is first related to several other basic studies, such as, Markov processes, queueing systems, queueing networks, Markov decision processes, sensitivity-based optimization, stochastic optimization, fluid and diffusion control. Therefore, our analysis begins from three simple introductions:1 Markov processes and Markov decision processes, queues and queueing networks, and queueing dynamic control. (a) Markov processes and Markov decision processesThe Markov processes, together with the Markov property, were first introduced by a Russian mathematician: Andrei Andreevich Markov in 1906. See Markov [238] for more details. From then on, as a basically mathematical tool, the Markov processes have extensively been discussed by many authors, e.g., see some excellent books by Doob [99], Karlin [175], Karlin and Taylor [176], Chung [80], Anderson [21], Kemeny et al. [181], Meyn and Tweedie [241], Chen [77], Ethier and Kurtz [110] and so on. In 1960, Howard [165] is the first to propose and discuss the MDP (or stochastic dynamic programming) in terms of his Ph.D thesis, which opened up a new and important field through an interesting intersection between Markov processes and dynamic programming (e.g., see Bellman and Kalaba [32]). From then on, not only are the MDPs an important branch in the area of Markov processes, but also it is a basic method in modern dynamic control theory. Crucially, the MDPs have been greatly motivated and widely applied in many practical areas in the past 60 years. Readers may refer to some excellent books, for example, the discrete-time MDPs by Puterman [261], Glasserman and Yao [143], Bertsekas [33], Bertsekas and Tsitsiklis [34], Hernádez-Lerma and Lasserre [155, 156], Altman [9], Koole [193] and Hu and Yue [166]; the continuous-time MDPs by Guo and Hernández-Lerma [145]; the partially observable MDPs by Cassandra [67] and Krishnamurthy [196]; the competitive MDPs (i.e., stochastic game) by [127]; the sensitivity-based optimization by Cao [58]; some applications of MDPs by Feinberg and Shwartz (Eds.) [122]and Boucherie and Van Dijk (Eds.) [44]; and so on. (b) Queues and queueing networksIn the early 20th century, a Danmark mathematician: Agner Krarup Erlang, published a pioneering work [109] of queueing theory in 1909, which started the study of queueing theory and traffic engineering. Over the past 100 years, queueing theory has been regarded as a key mathematical tool not only for analyzing practical s...

show abstract

“…We consider only a subset of the operators and the properties to be preserved by these operators. Extending different monotonicity results of the value functions to a multidimensional setting for a larger set of operators is an interesting and challenging problem even for the standard setting (see [21] for some recent results). Obviously, this presents a much bigger challenge in a robust formulation.…”

Section: Monotonicity Properties In Multidimensional Systemsmentioning

confidence: 99%

Structural properties of a class of robust inventory and queueing control problems

Turgay

Karaesmen

Örmeci

2017

Naval Research Logistics

Self Cite

View full text Add to dashboard Cite

In standard stochastic dynamic programming, the transition probability distributions of the underlying Markov Chains are assumed to be known with certainty. We focus on the case where the transition probabilities or other input data are uncertain. Robust dynamic programming addresses this problem by defining a min-max game between Nature and the controller. Considering examples from inventory and queueing control, we examine the structure of the optimal policy in such robust dynamic programs when event probabilities are uncertain. We identify the cases where certain monotonicity results still hold and the form of the optimal policy is determined by a threshold. We also investigate the marginal value of time and the case of uncertain rewards.

show abstract

Effects of System Parameters on the Optimal Cost and Policy in a Class of Multidimensional Queueing Control Problems

Cited by 9 publications

References 20 publications

Optimal Production Control of Remanufacturing Systems Based on a Queueing Model With Returns

Optimal Production Control of Remanufacturing Systems Based on a Queueing Model With Returns

An Overview for Markov Decision Processes in Queues and Networks

Structural properties of a class of robust inventory and queueing control problems

Contact Info

Product

Resources

About