Sequential finite-horizon Choquet-expected decision problems with uncertainty aversion

Abstract

This paper proposes a finite-horizon, sequential decision problem formulation where the probability measures used in Markov decision processes are replaced by a class of capacity measures. Subjective probability, arising for example in risk assessment carried out by humans, and modeling uncertainty, such as imprecise probability, can be represented by capacity measures. The aggregation operator employed to formulate the criterion is the so-called Choquet integral. A recursive equation is derived by applying results from sequential, stochastic, zero-sum games and cores of convex capacity. The recursion is applied to the finite-horizon control of Markovian jump linear systems involving a capacity measure. We show that a suboptimal solution, expressed as a Riccati equation, can be obtained by approximating the computation of the Choquet integral.