How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

Abstract

Endowing robots with the capability of assessing risk and making riskaware decisions is widely considered a key step toward ensuring safety for robots operating under uncertainty. But, how should a robot quantify risk? A natural and common approach is to consider the framework whereby costs are assigned to stochastic outcomes-an assignment captured by a cost random variable. Quantifying risk then corresponds to evaluating a risk metric, i.e., a mapping from the cost random variable to a real number. Yet, the question of what constitutes a "good" risk metric has received little attention within the robotics community. The goal of this paper is to explore and partially address this question by advocating axioms that risk metrics in robotics applications should satisfy in order to be employed as rational assessments of risk. We discuss general representation theorems that precisely characterize the class of metrics that satisfy these axioms (referred to as distortion risk metrics), and provide instantiations that can be used in applications. We further discuss pitfalls of commonly used risk metrics in robotics, and discuss additional properties that one must consider in sequential decision making tasks. Our hope is that the ideas presented here will lead to a foundational framework for quantifying risk (and hence safety) in robotics applications.

A Framework for Time-Consistent, Risk-Sensitive Model Predictive Control: Theory and Algorithms

“…Additional examples include semi-deviation measures, comonotonic risk measures, spectral risk, and optimized certainty equivalent; see [37] for further examples. The key point is that polytopic risk measures cover a full gamut of risk assessments, ranging from risk-neutral to worst case.…”

“…. , cardinality U poly,V (p) }, inequality(13)can be be rewritten as Now since Q = Q 0 and R = R 0, we also have the following identity:Then by Lemma C.1, inequalities(37) and(38) are equivalent to the existence of a matrix G that satisfies the following inequality for all l ∈ {1, . .…”

Bi-Directional Value Learning for Risk-Aware Planning Under Uncertainty

“…In our risk metric discussion, we follow definitions in [11], [12]. Accordingly, our risk metric falls in the category of risk for sequential decision making with deterministic policies, satisfying time-consistency (see [11], [12] for details). Specifically, we formalize the risk by compounding the failure probability,…”

Risk-sensitive inverse reinforcement learning via semi- and non-parametric methods

“…A4 reflects the intuition that a risk-averse agent should prefer to diversify . We refer the reader to Artzner et al (1999) and Majumdar and Pavone (2017) for a thorough justification of these axioms. In the following, we provide a hallmark example of CRMs, the conditional value-at-risk (CVaR) at level α ∈ ( 0 , 1 ] .…”

“…The limitations of EU theory in modeling human behavior has prompted substantial work on various alternative theories such as rank-dependent EU (Quiggin, 1982), expected uncertain utility (Gul and Pesendorfer, 2014), dual theory of choice (distortion risk measures) (Yaari, 1987), prospect theory (Barberis, 2013; Kahneman and Tversky, 1979), and many more (see Majumdar and Pavone (2017) for a recent review of the various axiomatic underpinnings of these risk measures). Further, one way to interpret the Ellsberg paradox is that humans are not only risk averse, but are also ambiguity averse : an observation that has sparked an alternative set of literature in decision theory on “ambiguity-averse” modeling; see, e.g., the recent review by Gilboa and Marinacci (2016).…”