Multi-cost Bounded Reachability in MDP

Hartmanns, Arnd; Junges, Sebastian; Katoen, Joost-Pieter; Quatmann, Tim

doi:10.1007/978-3-319-89963-3_19

Cited by 24 publications

(25 citation statements)

References 47 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further properties include temporal logic formulas based on PCTL [66] and CSL [8,11], conditional probability and cost queries [13,14], long-run average values [6,28,44] (also known as steady-state or mean payoff values), cost-bounded properties [69] (see Sect. 4.2), and support for multi-objective queries [51,109] (see Sect.…”

Section: Propertiesmentioning

confidence: 99%

The probabilistic model checker Storm

Hensel

Junges

Katoen

et al. 2021

Int J Softw Tools Technol Transfer

Self Cite

View full text Add to dashboard Cite

We present the probabilistic model checker Storm. Storm supports the analysis of discrete- and continuous-time variants of both Markov chains and Markov decision processes. Storm has three major distinguishing features. It supports multiple input languages for Markov models, including the Jani and Prism modeling languages, dynamic fault trees, generalized stochastic Petri nets, and the probabilistic guarded command language. It has a modular setup in which solvers and symbolic engines can easily be exchanged. Its Python API allows for rapid prototyping by encapsulating Storm’s fast and scalable algorithms. This paper reports on the main features of Storm and explains how to effectively use them. A description is provided of the main distinguishing functionalities of Storm. Finally, an empirical evaluation of different configurations of Storm on the QComp 2019 benchmark set is presented.

show abstract

Section: Propertiesmentioning

confidence: 99%

The probabilistic model checker Storm

Hensel

Junges

Katoen

et al. 2021

Int J Softw Tools Technol Transfer

Self Cite

View full text Add to dashboard Cite

show abstract

“…For MDP, step-and rewardbounded reachability probabilities can be converted to total reward objectives by unfolding the current amount of steps (or rewards) into the state-space of the model. Approaches that avoid such an expensive unfolding have been presented in [28] for objectives with step-bounds and in [34,35] for objectives with one or multiple reward-bounds. Time-bounded reachability probabilities for MA have been considered in [47].…”

Section: Combining Long-run Average and Total Rewardsmentioning

confidence: 99%

Multi-objective Optimization of Long-run Average and Total Rewards

Quatmann

Katoen

2021

Tools and Algorithms for the Construction and Analysis of Systems

Self Cite

View full text Add to dashboard Cite

This paper presents an efficient procedure for multi-objective model checking of long-run average reward (aka: mean pay-off) and total reward objectives as well as their combination. We consider this for Markov automata, a compositional model that captures both traditional Markov decision processes (MDPs) as well as a continuous-time variant thereof. The crux of our procedure is a generalization of Forejt et al.’s approach for total rewards on MDPs to arbitrary combinations of long-run and total reward objectives on Markov automata. Experiments with a prototypical implementation on top of the Storm model checker show encouraging results for both model types and indicate a substantial improved performance over existing multi-objective long-run MDP model checking based on linear programming.

show abstract

“…Surveys on multi-objective decision making in AI and machine learning can be found in [51] and [58], respectively. This article is an extended version of a previous conference paper [32]. We provide more details on the core algorithms, extended proofs, an expanded explanation of our visualisations, and additional models in the experimental evaluation.…”

Section: Related Workmentioning

confidence: 99%

Multi-cost Bounded Tradeoff Analysis in MDP

et al. 2020

Self Cite

View full text Add to dashboard Cite

We provide a memory-efficient algorithm for multi-objective model checking problems on Markov decision processes (MDPs) with multiple cost structures. The key problem at hand is to check whether there exists a scheduler for a given MDP such that all objectives over cost vectors are fulfilled. We cover multi-objective reachability and expected cost objectives, and combinations thereof. We further transfer approaches for computing quantiles over single cost bounds to the multi-cost case and highlight the ensuing challenges. An empirical evaluation shows the scalability of our new approach both in terms of memory consumption and runtime. We discuss the need for more detailed visual presentations of results beyond Pareto curves and present a first visualisation approach that exploits all the available information from the algorithm to support decision makers.

show abstract

Multi-cost Bounded Reachability in MDP

Cited by 24 publications

References 47 publications

The probabilistic model checker Storm

The probabilistic model checker Storm

Multi-objective Optimization of Long-run Average and Total Rewards

Multi-cost Bounded Tradeoff Analysis in MDP

Contact Info

Product

Resources

About