Stationary optimal policies in a class of multichain positive dynamic programs with finite state space and risk-sensitive criterion

Abstract

Abstract. This work concerns Markov decision processes with finite state space and compact action sets. The decision maker is supposed to have a constant-risk sensitivity coefficient, and a control policy is graded via the risk-sensitive expected total-reward criterion associated with nonnegative one-step rewards. Assuming that the optimal value function is finite, under mild continuity and compactness restrictions the following result is established: If the number of ergodic classes when a stationary policy is used to drive the system depends continuously on the policy employed, then there exists an optimal stationary policy, extending results obtained by Schäl (1984) for risk-neutral dynamic programming. We use results recently established for unichain systems, and analyze the general multichain case via a reduction to a model with the unichain property.

About stability of risk-seeking optimal stopping

“…The proof of (5.5) is accomplished as the proof of Lemma 3 in [6]. Equation (5.6) follows from (5.1), (5.2), (5.5) and the fact that V * (Θ) = 1.…”

Reducibilidad en matrices finitas no negativas con aplicaciones a economía y control estocástico

“…La primera aplicación será en Economía y estará relacionada con la Matriz de Insumo-Producto de Leontief [28]. La segunda aplicación será a los procesos de control de Markov con recompensa total esperada, casos neutral y sensible al riesgo [4], [5], [6], la cual está motivada por un problema de juego de apuestas planteado por Sheldon Ross en su libro "Introduction to Stochastic Dynamic Programming"( [21]).…”

Juegos estocásticos transitorios con recompensa total

“…En contraste con los modelos no controlados considerados en la sección 3.1 y su extensión a los modelos controlados en la sección 3.2 nosotros suponemos que la utilidad esperada U π i (γ, n) depende de las decisiones f (1),n , f (2),n tomadas por los dos, en el caso de un sólo jugador este tipo de políticas son abordadas en [12,13,59]. Recuérdese que:…”