Multi-cost Bounded Tradeoff Analysis in MDP

Abstract

We provide a memory-efficient algorithm for multi-objective model checking problems on Markov decision processes (MDPs) with multiple cost structures. The key problem at hand is to check whether there exists a scheduler for a given MDP such that all objectives over cost vectors are fulfilled. We cover multi-objective reachability and expected cost objectives, and combinations thereof. We further transfer approaches for computing quantiles over single cost bounds to the multi-cost case and highlight the ensuing challenges. An empirical evaluation shows the scalability of our new approach both in terms of memory consumption and runtime. We discuss the need for more detailed visual presentations of results beyond Pareto curves and present a first visualisation approach that exploits all the available information from the algorithm to support decision makers.

The probabilistic model checker Storm

“…These costs are attached in the model, and then, one may analyze cost-bounded reachability with the adequate query. The [69,70], generalizing ideas from [59,89] to multiple cost dimensions. The reduced memory footprint allows to handle much larger models, and often the reduced memory consumption also yields faster verification times.…”

“…Cost-bounded reachability is closely related to quantile properties [70,89], where one fixes a desired reachability probability and asks how many resources have to be invested in order to achieve this probability.…”

Multi-objective Optimization of Long-run Average and Total Rewards

“…[31] considers LTL formulae with probability thresholds while maximizing an expected LRA reward. [35,41] address multi-objective quantiles on reachability properties while [50,20] consider multi-objective combinations of percentile queries on MDP and LRA objectives. [6] treats resilient systems ensuring constraints on the repair mechanism while maximizing the expected LRA reward when being operational.…”

“…For MDP, step-and rewardbounded reachability probabilities can be converted to total reward objectives by unfolding the current amount of steps (or rewards) into the state-space of the model. Approaches that avoid such an expensive unfolding have been presented in [28] for objectives with step-bounds and in [34,35] for objectives with one or multiple reward-bounds. Time-bounded reachability probabilities for MA have been considered in [47].…”

“…Multi-objective MDP Various types of objectives known from conventionalsingle-objective-model checking have been lifted to the multi-objective case. These objectives range over ω-regular specifications including LTL [26,27], expected (discounted and non-discounted) total rewards [21,27,28,52,22], stepbounded and reward-bounded reachability probabilities [28,35], and-most relevant for this work-expected long-run average (LRA) rewards [18,11,20], also known as mean pay-offs. For the latter, all current approaches build upon linear programming (LP) which yields a theoretical time-complexity polynomial in the model size.…”