Trade-off analysis meets probabilistic model checking

Abstract

Probabilistic model checking (PMC) is a well-established and powerful method for the automated quantitative analysis of parallel distributed systems. Classical PMC-approaches focus on computing probabilities and expectations in Markovian models annotated with numerical values for costs and utility, such as energy and performance. Usually, the utility gained and the costs invested are dependent and a trade-off analysis is of utter interest.In this paper, we provide an overview on various kinds of nonstandard multi-objective formalisms that enable to specify and reason about the trade-off between costs and utility. In particular, we present the concepts of quantiles, conditional probabilities and expectations as well as objectives on the ratio between accumulated costs and utility. Such multi-objective properties have drawn very few attention in the context of PMC and hence, there is hardly any tool support in state-of-the-art model checkers. Furthermore, we broaden our results towards combined quantile queries, computing conditional probabilities those conditions are expressed as formulas in probabilistic computation tree logic, and the computation of ratios which can be expected on the long-run.

Conditional Value-at-Risk for Reachability and Mean Payoff in Markov Decision Processes

“…Quantile queries for more complex constraints have also been considered, namely their conjunctions [9,21], conjunctions with expectations [15] or generally Boolean expressions [26]. Some of these approaches have already been practically applied and found useful by domain experts [4,5].…”

Safety-Constrained Reinforcement Learning for MDPs

“…A reachability property P ≤λ (♦T ) with upper probability bound λ ∈ [0, 1] ⊆ Q and target set T ⊆ S constrains the probability to finally reach T from s I in M to be at most λ. Analogously, expected cost properties E ≤κ (♦G) impose an upper bound κ ∈ Q on the expected cost to reach goal states G ⊆ S. Combining both typesprovid of properties, the intuition is that a set of bad states T shall only be reached with a certain probability λ (safety specification) while the expected cost for reaching a set of goal states G has to be below κ (performance specification). This can be verified using multi-objective model checking [6,7,8], provided all problem data (i.e., probabilities and costs) are a-priori known.…”

“…Finally, there might be safe randomized schedulers that induce better optimal costs than all deterministic schedulers [7,8]. To compute randomized permissive schedulers, the difficulty is that there are arbitrarily (or even infinitely) many probability distributions over actions.…”

“…Given an MDP with a cost structure, synthesize an optimal policy subject to safety constraints. This multi-objective model checking problem is studied in [6,7,8]. But what if the cost function is not known?…”

Multi-weighted Markov Decision Processes with Reachability Objectives

“…It allows to express various (quantitative or qualitative) properties over the model, and to synthesize strategies accordingly. This new field of research is very rich and ambitious, with various types of objective combinations (see for instance [2,18] for recent overviews). For recent developments on MDPs, one can cite:…”