Multilevel Optimization Modeling for Risk-Averse Stochastic Programming

Abstract

Abstract. Coherent risk measures have become a popular tool for incorporating risk aversion into stochastic optimization models. For dynamic models in which uncertainty is resolved at more than one stage, however, using coherent risk measures within a standard single-level optimization framework becomes problematic. To avoid severe time-consistency difficulties, the current state of the art is to employ risk measures of a specific nested form, which unfortunately have some undesirable and somewhat counterintuitive modeling properties. For one thing, this technique requires increasing risk aversion as risks and reward recede in time. Further, it produces objective functions that cannot be law invariant with respect to the total incurred costs and rewards, meaning that two solutions with identical probability distributions of final wealth may be assigned different levels of risk, and the nested form of the objective function cannot be simplified. These properties deter practical acceptance of such models, and are particularly undesirable for situations with close final time horizons. This paper summarizes these issues and then presents an alternative multilevel optimization modeling approach that enforces a form of time consistency through constraints rather than by restricting the modeler's choice of objective function. This technique leads to models that are time-consistent even while using time-inconsistent risk measures, and can easily be formulated to be law invariant with respect to the final wealth if so desired. We argue that this approach should be the starting point for all multistage optimization modeling. When used with time-consistent objective functions, we show its multilevel optimization constraints become redundant and the associated models thus simplify to a more familiar single-objective form. Unfortunately, this paper also shows that its proposed approach leads to N Phard models, even in the simplest imaginable setting in which it would be needed: three-stage linear problems on a finite probability space, using the standard meansemideviation and average-value-at-risk measures. Finally, we show that for a simple but reasonably realistic test application, that the kind of models we propose, while being drawn from an N P-hard family and certainly more time consuming to solve than those obtained from the nested-objective approach, are readily solvable to global optimality using a standard commercial MILP solver. Therefore, there seems some promise of the modeling approach being useful despite its computational complexity properties.Acknowledgements: This work was supported by a grant from the Air Force Office of Scientific Research, award FA9550-11-1-0164. The authors would like to thank Andrzej Ruszczyński for his assistance and support.Note: Sections 4-7 of this report essentially duplicate material presented in the earlier reports RRR 7-2013 and (using different notation) RRR 17-2012. The remaining sections contain extensive new material and results.

Alleviating time inconsistent behaviors via a competition scheme

“…We propose a winner-takes-all competition between two salespersons, under which the winner can receive a bonus at the end of sale period, and the 1 Besides the quasi-hyperbolic discounting function, a statedependent or non-separable objective function may also generate time inconsistency phenomena. More specifically, the nonseparability of the objective function causes the time inconsistency issue in [2,6,13]; the non-separability of the variance and the state-dependent risk aversion cause the time inconsistency issue in [3,8,9]; the presence of probability weighting in the objective function causes the time inconsistency issue in [33]. 2 In control theory, the time inconsistency issue only arises in a dynamic setting.…”

“… Besides the quasi‐hyperbolic discounting function, a state‐dependent or non‐separable objective function may also generate time inconsistency phenomena. More specifically, the non‐separability of the objective function causes the time inconsistency issue in ; the non‐separability of the variance and the state‐dependent risk aversion cause the time inconsistency issue in ; the presence of probability weighting in the objective function causes the time inconsistency issue in . …”

A unified algorithm framework for mean-variance optimization in discounted Markov decision processes

“…Variance is a quadratic function of mean, and a variance-related problem is equivalent to an MDP with a special reward function, whose value for each state depends on policy instead of action, i.e., the variance value function at a current state will be affected by actions chosen at not only the current epoch but also future epochs. This dependency deprives the time-consistency property and revokes traditional dynamic programming (DP) methods (Puterman 2005, Eckstein et al 2016, Bisi et al 2020. In other words, the Bellman optimality equation does not optimize over the admissible action set for a state x (max a∈A(x) ), but over the policy space for the whole state space S (max d∈D ), and we can not divide and conquer this problem in a traditional manner.…”

A bilevel groundwater management model with minimization of stochastic health risks at the leader level and remediation cost at the follower level

“…Moreover, the uncertainties of health risk would have significant influences upon the optimal strategies. Furthermore, most of the previous BLP problems were mainly suitable for smallscale problems due to the complexity in the solution method (Mehlitz and Wachsmuth 2016;Hemmati and Smith 2016;Eckstein et al 2016). Therefore, this study aims to fill this gap by proposing a new bilevel programming model with leader-level environmental and followerlevel economic goals.…”