On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue

Abstract

The derivation of structural properties for unbounded jump Markov processes cannot be done using standard mathematical tools, since the analysis is hindered due to the fact that the system is not uniformizable. We present a promising technique, a smoothed rate truncation method, to overcome the limitations of standard techniques and allow for the derivation of structural properties. We introduce this technique by application to a processor sharing queue with impatient customers that can retry if they renege. We are interested in structural properties of the value function of the system as a function of the arrival rate.

Efficient Content Delivery in the Presence of Impatient Jobs

“…However, we will observe in Section IV that the result in [2] allows structural results to be proven for Problem P. We will show that in both settings, µ = ∞ and µ < ∞, monotone policies are average optimal. That is, there exists a threshold H for which the system prescribes not to take jobs into service for all states m ≤ H − 1, and serves them otherwise.…”

“…However, obtaining structural results for nonuniformizable problems is much more involved. In a recent study, the authors in [2] have developed a methodology to overcome the problem originated by unbounded jump Markov processes. This method is known as the Smoothed Rate Truncation method (SRT) and consists of approximating the infinite state Markov Decision Process (MDP) by finite state MDPs.…”

“…The presence of abandonments in the queue causes the system to be non uniformizable, and hence proving optimality of monotone policies becomes extremely challenging. In a recent work [2], the authors have been able to overcome this issue by solving a truncated version of the original problem and taking the limit as the truncating parameter grows to infinity. This method has successfully been applied in [2] for a retrial queue and in [3] for a multiclass abandonment queue.…”

“…In a recent work [2], the authors have been able to overcome this issue by solving a truncated version of the original problem and taking the limit as the truncating parameter grows to infinity. This method has successfully been applied in [2] for a retrial queue and in [3] for a multiclass abandonment queue. In our second contribution we analyze the stationary behavior of the system and compute the steady-state job-length distribution for all µ ∈ R + ∪ {∞}, using a generating function approach.…”

Bloody fast blood collection

“…Their advised course of action for an continuous-time MDP with unbounded rates is to apply some perturbation and then apply uniformization. Bhulai et al [27] introduce the first general method, Smoothed Rate Truncation (SRT), for this perturbation that conserves the structural properties of the original model. SRT is based on linear smoothing of unbounded rates to obtain a finite set of recurring states.…”