Qualitative Reachability for Open Interval Markov Chains

Abstract

Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals, previous work has considered also open interval Markov chains, in which the intervals can also be open or halfopen.In this paper we focus on qualitative reachability problems for open interval Markov chains, which consider whether the optimal (maximum or minimum) probability with which a certain set of states can be reached is equal to 0 or 1. We present polynomial-time algorithms for these problems for both of the standard semantics of interval Markov chains. Our methods do not rely on the closure of open intervals, in contrast to previous approaches for open interval Markov chains, and can characterise situations in which probability 0 or 1 can be attained not exactly but arbitrarily closely.

Probabilistic Timed Automata with One Clock and Initialised Clock-Dependent Probabilities

“…The theorem follows from Lemma 4.2, Lemma 4.5, Lemma 4.7, Proposition 2.2, the fact that the IMDP defined in this section can be constructed in polynomial time, and the fact that quantitative and qualitative problems for IMDPs can be solved in polynomial time, given that there exist polynomial-time algorithms for analogous problems on IMCs with the semantics adopted in this paper [PLSS13,CHK13,CK15,Spr18]. We add that the quantitative and qualitative problems for 1c-cdPTAs are PTIME-hard, following from the PTIME-hardness of the corresponding problems for MDPs [PT87, CDH10].…”

“…Computing P max [[C]],s (♦T ) and P min [[C]],s (♦T ) can be done for an IMC C simply by transforming the IMC by closing all of its (half-)open intervals, then employing a standard maximum/minimum reachability probability computation on the new, "closed" IMC (for example, the algorithms of [SVA06,CHK13]): the correctness of this approach is shown in [CK15]. Algorithms for qualitative problems of IMCs (with open, half-open and closed intervals) are given in [Spr18]. All of the aforementioned algorithms run in polynomial time in the size of the IMC, which is obtained as the sum over all states s, s ∈ S of the binary representation of the endpoints of P(s, s ), where rational numbers are encoded as the quotient of integers written in binary.…”

“…In contrast to the standard formulation of interval Markov decision processes, we allow edges corresponding to probabilistic choices to be labelled not only with closed intervals, but also with open and half-open intervals. While (half-)open intervals have been considered previously in the context of interval Markov chains in [CK15,Spr18], we are unaware of any work considering them in the context of interval Markov decision processes. Open intervals in the constructed interval Markov decision process provide a natural way of representing strict constraints on clocks in the one-clock clock-dependent probabilistic timed automaton.…”

“…We proceed by giving some preliminary concepts in Section 2: this includes a reduction from interval Markov decision processes to interval Markov chains [JL91, KU02,SVA06] with the standard Markov decision process-based semantics, which may be of independent interest. The reduction takes open and half-open intervals into account; while [CK15] has shown that open interval Markov chains can be reduced to closed Markov chains for the purposes of quantitative properties, [Spr18] shows that the open/closed distinction is critical for the evaluation of qualitative properties. In Section 3, we present the definition of one-clock clock-dependent probabilistic timed automata, and describe the transformation to interval Markov decision processes in Section 4.…”

The complexity of reachability in parametric Markov decision processes

“…Often, probability distributions in these models are difficult to assess precisely during design time of a system. This shortcoming has led to interval MCs [15,35,50,54] and interval MDPs (also known as Bounded-parameter MDPs) [27,42,58], which allow for interval-labelled transitions. Analysis under interval Markov models is often too pessimistic:…”