We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost function may be unbounded above. The optimization problem is split into two minimization problems using an infimum representation for spectral risk measures. We show that the inner minimization problem can be solved as an ordinary MDP on an extended state space and give sufficient conditions under which an optimal policy exists. Regarding the infinite dimensional outer minimization problem, we prove the existence of a solution and derive an algorithm for its numerical approximation. Our results include the findings in Bäuerle and Ott (Math Methods Oper Res 74(3):361–379, 2011) in the special case that the risk measure is Expected Shortfall. As an application, we present a dynamic extension of the classical static optimal reinsurance problem, where an insurance company minimizes its cost of capital.
In this paper we consider reinsurance or risk sharing from a macroeconomic point of view. Our aim is to find socially optimal reinsurance treaties. In our setting we assume that there are n insurance companies each bearing a certain risk and one representative reinsurer. The optimization problem is to minimize the sum of all capital requirements of the insurers where we assume that all insurance companies use a form of Range-Value-at-Risk. We show that in case all insurers use Value-at-Risk and the reinsurer's premium principle satisfies monotonicity, then layer reinsurance treaties are socially optimal. For this result we do not need any dependence structure between the risks. In the general setting with Range-Value-at-Risk we obtain again the optimality of layer reinsurance treaties under further assumptions, in particular under the assumption that the individual risks are positively dependent through the stochastic ordering. Our results include the findings in [12] in the special case n = 1. At the end, we discuss the difference between socially optimal reinsurance treaties and individually optimal ones by looking at a number of special cases.
We consider robust Markov decision processes with Borel state and action spaces, unbounded cost, and finite time horizon. Our formulation leads to a Stackelberg game against nature. Under integrability, continuity, and compactness assumptions, we derive a robust cost iteration for a fixed policy of the decision maker and a value iteration for the robust optimization problem. Moreover, we show the existence of deterministic optimal policies for both players. This is in contrast to classical zero-sum games. In case the state space is the real line, we show under some convexity assumptions that the interchange of supremum and infimum is possible with the help of Sion’s minimax theorem. Further, we consider the problem with special ambiguity sets. In particular, we are able to derive some cases where the robust optimization problem coincides with the minimization of a coherent risk measure. In the final section, we discuss two applications: a robust linear-quadratic problem and a robust problem for managing regenerative energy.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.