Decision-making under uncertainty: beyond probabilities

Abstract

This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes beyond this classical interpretation, particularly by employing a clear distinction between aleatoric and epistemic uncertainty. The paper features an overview of Markov decision processes (MDPs) and extensions to account for partial observability and adversarial behavior. These models sufficiently capture aleatoric uncertainty, but fail to account for epistemic uncertainty robustly. Consequently, we present a thorough overview of so-called uncertainty models that exhibit uncertainty in a more robust interpretation. We show several solution techniques for both discrete and continuous models, ranging from formal verification, over control-based abstractions, to reinforcement learning. As an integral part of this paper, we list and discuss several key challenges that arise when dealing with rich types of uncertainty in a model-based fashion.

Probabilistic Net Load Forecasting Framework for Application in Distributed Integrated Renewable Energy Systems

“…For this purpose, detailed reviews of known techniques, application examples, and challenges have been published [31,28]. In addition to the metrics, it is also necessary to determine whether the uncertainty is aleatoric or epistemic [32]. There are a few examples of applications in which multiple sectors are taken into account.…”

Advanced Power Converters and Learning in Diverse Robotic Innovation: A Review

“…Robots can also learn from human-like behavior and adapt using RL [184,185]. Lastly, handling uncertainty in the environment is important and Bayesian RL helps robots make decisions while considering potential risks [186][187][188][189][190][191][192][193]. All of this has transformed how robots work in different fields, from manipulation to agile movement, making them smarter and more adaptable.…”

Explanation Paradigms Leveraging Analytic Intuition (ExPLAIn)

“…In their paper "Decision-making under uncertainty: beyond probabilities. Challenges and Perspectices" [4], the authors give an introductory overview of the field of decision making under uncertainty with a special focus on the different ways in which aleatoric and epistemic uncertainty are mathematically formalized. Furthermore, the authors introduce robust reinforcement learning [32], which seeks to not only make good decisions, but decisions that are in some respect safe, and bayesian reinforcement learning [15,26], which provides a natural mechanism for the modeling of epistemic uncertainty in bayesian prior distributions.…”