Discrete-Time Rewards Model-Checked

Abstract

This paper presents a model-checking approach for analyzing discrete-time Markov reward models. For this purpose, the temporal logic probabilistic CTL is extended with reward constraints. This allows to formulate complex measures-involving expected as well as accumulated rewards-in a precise and succinct way. Algorithms to efficiently analyze such formulae are introduced. The approach is illustrated by model-checking a probabilistic cost model of the IPv4 zeroconf protocol for distributed address assignment in ad-hoc networks.

Stochastic Model Checking

“…Below are some typical examples of CSL formulae: 4.5] served ] -the probability that a request is served within the first 4.5 seconds is greater than 0.9; -P 0.1 [3 [10,∞) error ] -the probability that an error occurs after 10 seconds of operation is at most 0.1; -down → P >0.75 [¬fail U [1,2] up ] -when a shutdown occurs, the probability of system recovery being completed in between 1 and 2 hours without further failures occurring is greater than 0.75; -S <0.01 [insufficient routers] -in the long-run, the probability that an inadequate number of routers are operational is less than 0.01.…”

Bisimulation Minimisation Mostly Speeds Up Probabilistic Model Checking

“…Let AP be a fixed, finite set of atomic propositions. A (labelled) DTMC D is a tuple (S, P, L) where S is a finite set of states, P : S × S → [0, 1] is a probability matrix such that s ∈S P(s, s ) = 1 for all s ∈ S, and L : S → 2 AP is a labelling function which assigns to each state s ∈ S the set L(s) of atomic propositions that hold in s. A path through a DTMC is a sequence 1 The logic PCTL. Let a ∈ AP, probability p ∈ [0, 1], k ∈ N (or k = ∞) and be either ≤ or ≥.…”

“…The two main operators that extend PCTL to Probabilistic Reward CTL (PRCTL) [1] are P p (Φ U ≤k ≤r Ψ ) and E =k ≤r (Φ). The until-operator is equipped with a bound on the maximum number (k) of allowed hops to reach the goal states, and a bound on the maximum allowed cumulated reward (r) before reaching these states.…”

“…The sets of states U 0 and U 1 can be collapsed into single states u 0 and u 1 , respectively. This results in the initial partition { {u 0 }, {u 1 …”

“…For DMRMs, these formulas are checked using a path graph generation algorithm as proposed in [1] which has a time complexity in O(k·r·|S| 3 ), where k and r are the time-bound and reward-bound, respectively. For CMRMs, we employed the discretization approach by Tijms and Veldman as proposed in [21] which runs in time O(t·r·|S| 3 ·d −2 ) where d is the step size of the discretisation.…”