Monotonicity in Markov Reward and Decision Chains: Theory and Applications

Technical Note—Optimal Structural Results for Assemble-to-Order Generalized M-Systems

“…A standard approach for studying the optimal policies of MDPs is to explore the first-and/or second-order properties of the optimal cost function (see Koole 2006). Optimal cost functions for multivariate MDPs (like ours) are typically shown to be convex in each dimension of the state space.…”

Structural properties of the optimal resource allocation policy for single-queue systems

“…For both models, we show via dynamic programming that (i) the optimal allocation policy has a work-conservation property that implies when the system is not empty, the optimal policy is not allowed to keep all computing resources idle, (ii) the optimal number of servers follows a step function with as extreme policy the bang-bang control policy, which means a facility receives all computing resources or none at all, and moreover (iii) we also provide the conditions under which the bang-bang control policy is optimal. The techniques to prove such results are based on monotonicity properties of the dynamic programming relative value function (see, e.g., Koole 1998Koole , 2006Rykov 2001).…”

Spare Parts Planning and Control for Maintenance Operations

“…To analyze the value function V n (x, y) in §4.4.1.3, we employ the event based dynamic programming approach introduced by Koole (1998Koole ( , 2006. To this end, let V denote the set of all functions v : S → R and let f, f 1 , ..., f m+2 ∈ V. We define the following operators…”

“…These operators are variations to operators defined by Koole (1998Koole ( , 2004Koole ( , 2006 and are originally intended to model various common queueing mechanisms such as arrival control (T AC(i) ), transfer departures from multi-server tandem queues (T TD(i) ), and departures from multi-server queues (T D(i) ), while the operators T cost f (x, y), T env and T unif are mainly convenient for bookkeeping. The Bellman recursion for our MDP, (4.4), can now be written succinctly as…”

“…UI is the set of upstream increasing functions as introduced in Koole (2004) and renamed in Koole (2006). The set SM(i, j) consists of functions with a specific supermodularity property.…”