Reachability in Augmented Interval Markov Chains

Tools and Algorithms for the Construction and Analysis of Systems

et al. 2021

This paper presents a novel method for the automated synthesis of probabilistic programs. The starting point is a program sketch representing a finite family of finite-state Markov chains with related but distinct topologies, and a reachability specification. The method builds on a novel inductive oracle that greedily generates counter-examples (CEs) for violating programs and uses them to prune the family. These CEs leverage the semantics of the family in the form of bounds on its best- and worst-case behaviour provided by a deductive oracle using an MDP abstraction. The method further monitors the performance of the synthesis and adaptively switches between inductive and deductive reasoning. Our experiments demonstrate that the novel CE construction provides a significantly faster and more effective pruning strategy leading to an accelerated synthesis process on a wide range of benchmarks. For challenging problems, such as the synthesis of decentralized partially-observable controllers, we reduce the run-time from a day to minutes.

Section: Related Workmentioning

confidence: 99%

Inductive Synthesis for Probabilistic Programs Reaches New Horizons

Andriushchenko

Češka

Tools and Algorithms for the Construction and Analysis of Systems

et al. 2021

“…Pr sI →T M (u) ≥ λ? This problem is square-rootsum hard [14]. For any fixed number of parameters, this problem is decidable in P [30].…”

Section: Example 2 For the Pmc Inmentioning

confidence: 99%

“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”

Section: Example 2 For the Pmc Inmentioning

confidence: 99%

See 1 more Smart Citation

Are Parametric Markov Chains Monotonic?

Spel

Automated Technology for Verification and Analysis

Katoen

2019

This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct-only using the graph structure of the Markov chain and local transition probabilities-a pre-order on the states. Our algorithm cheaply checks a sufficient condition for monotonicity. Experiments show that monotonicity in several benchmarks is automatically detected, and monotonicity can speed up parameter synthesis up to orders of magnitude faster than a symbolic baseline.

“…The theorem even holds for almost-sure reachability properties. The proof is a straightforward adaption of results for augmented interval Markov chains [17,Theorem 3], partial information games [15], or partially observable MDPs [14].…”

Section: Theorem 1 the Feasibility Synthesis Problem Is Np-completementioning

confidence: 99%

Shepherding Hordes of Markov Chains

Češka

Jansen

Lecture Notes in Computer Science

et al. 2019

This paper considers large families of Markov chains (MCs) that are defined over a set of parameters with finite discrete domains. Such families occur in software product lines, planning under partial observability, and sketching of probabilistic programs. Simple questions, like 'does at least one family member satisfy a property?', are NP-hard. We tackle two problems: distinguish family members that satisfy a given quantitative property from those that do not, and determine a family member that satisfies the property optimally, i.e., with the highest probability or reward. We show that combining two well-known techniques, MDP model checking and abstraction refinement, mitigates the computational complexity. Experiments on a broad set of benchmarks show that in many situations, our approach is able to handle families of millions of MCs, providing superior scalability compared to existing solutions.