Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal

Total Expected Discounted Reward MDPS : Existence of Optimal Policies

“…These assumptions correspond to two different types of convergence of probability measures: setwise convergence and weak convergence. Such assumptions, the so-called S and W were introduced by Schäl [17,18] for the case of compact action spaces. Since we start with possibly non-compact action sets, we shall use abbreviations Su and Wu, where the symbol "u" indicate that the sets A(x) can be unbounded and therefore non-compact.…”

“…Theorem 14 is Proposition 9.17 from Bertsekas and Shreve [2]. Schäl [17,18] introduced Assumptions S and W, where S stands for setwise continuity of transition probabilities and W stands for weak continuity. Theorem 16 is from Schäl [18].…”

“…Schäl [17,18] introduced Assumptions S and W, where S stands for setwise continuity of transition probabilities and W stands for weak continuity. Theorem 16 is from Schäl [18]. The monograph [13] by Hernández-Lerma and Lasserre primarily covers MDPs with setwise continuous transition probabilities.…”

Routing of Airplanes to Two Runways: Monotonicity of Optimal Controls

“…[5], [11] or [12] for a general discussion of dynamic programming). Here, we present only a simple type of dynamic programming model which is sufficient to cover the aircraft routing problem formulated in the next Section.…”

“…As our cost function is non-negative and the action space is finite, it is well known (see e.g. [12]) that an optimal policy exists, i.e. a policy δ * , with…”