Minimal Critical Subsystems for Discrete-Time Markov Models

Wimmer, Ralf; Jansen, Nils; Ábrahám, Erika; Becker, Bernd; Katoen, Joost-Pieter

doi:10.1007/978-3-642-28756-5_21

Cited by 36 publications

(33 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case the MDP will be reduced to a DTMC. Then several results exist to generate counterexample in DTMC, for example, k shortest path algorithm [28] or minimal critical subsystem generation from [29].…”

Section: A Counterexamplesmentioning

confidence: 99%

Permissive Supervisor Synthesis for Markov Decision Processes Through Learning

Zhang

Lin

2019

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

This paper considers the permissive supervisor synthesis for probabilistic systems modeled as Markov Decision Processes (MDP). Such systems are prevalent in power grids, transportation networks, communication networks and robotics. Unlike centralized planning and optimization based planning, we propose a novel supervisor synthesis framework based on learning and compositional model checking to generate permissive local supervisors in a distributed manner. With the recent advance in assume-guarantee reasoning verification for probabilistic systems, building the composed system can be avoided to alleviate the state space explosion and our framework learn the supervisors iteratively based on the counterexamples from verification. Our approach is guaranteed to terminate in finite steps and to be correct.

show abstract

Section: A Counterexamplesmentioning

confidence: 99%

Permissive Supervisor Synthesis for Markov Decision Processes Through Learning

Zhang

Lin

2019

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

show abstract

“…By converting DTMC M π into a weighted directed graph, counterexample can be found by solving a k-shortest paths (KSP) problem or a hop-constrained KSP (HKSP) problem [6]. Alternatively, counterexamples can be found by using Satisfiability Modulo Theory solving or mixed integer linear programming to determine the minimal critical subsystems that capture the counterexamples in M π [23]. A policy can also be synthesized by solving the objective min π P =?…”

Section: Pctl Model Checkingmentioning

confidence: 99%

“…PCTL model checking for DTMCs can be solved in time linear in the size of the formula and polynomial in the size of the state space [7]. Counterexample generation can be done either by enumerating paths using the k-shortest path algorithm or determining a critical subsystem using either a SM T formulation or mixed integer linear programming (MILP) [23]. For the k-shortest path-based algorithm, it can be computationally expensive sometimes to enumerate a large amount of paths (i.e.…”

mentioning

confidence: 99%

Safety-Aware Apprenticeship Learning

Zhou

2018

Computer Aided Verification

View full text Add to dashboard Cite

Abstract. Apprenticeship learning (AL) is a kind of Learning fromDemonstration techniques where the reward function of a Markov Decision Process (MDP) is unknown to the learning agent and the agent has to derive a good policy by observing an expert's demonstrations. In this paper, we study the problem of how to make AL algorithms inherently safe while still meeting its learning objective. We consider a setting where the unknown reward function is assumed to be a linear combination of a set of state features, and the safety property is specified in Probabilistic Computation Tree Logic (PCTL). By embedding probabilistic model checking inside AL, we propose a novel counterexample-guided approach that can ensure safety while retaining performance of the learnt policy. We demonstrate the effectiveness of our approach on several challenging AL scenarios where safety is essential.

show abstract

“…This sub-system can thus be viewed as another representation of the set of runs that all satisfy the property ϕ whose probability mass exceeds the required upper bound. In [10,11] we suggested to obtain minimal critical sub-systems. Here, minimality refers to the state space size of the sub-Markov chain.…”

mentioning

confidence: 99%

“…Here, minimality refers to the state space size of the sub-Markov chain. Whereas [6,9] use heuristic approaches to construct small (but not necessarily minimal) critical sub-systems, [10,11] advocates the use of mixed integer linear programming (MILP) [12]. The MILP-approach is applicable to ω-regular properties (that include reachability) for both Markov chains and Markov decision processes (MDPs) [11], which are a slightly variant of probabilistic automata.…”

mentioning

confidence: 99%

High-level Counterexamples for Probabilistic Automata

Wimmer¹,

Jansen²,

Ábrahám³

et al. 2015

Logical Methods in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Providing compact and understandable counterexamples for violated system properties is an essential task in model checking. Existing works on counterexamples for probabilistic systems so far computed either a large set of system runs or a subset of the system's states, both of which are of limited use in manual debugging. Many probabilistic systems are described in a guarded command language like the one used by the popular model checker PRISM. In this paper we describe how a smallest possible subset of the commands can be identified which together make the system erroneous. We additionally show how the selected commands can be further simplified to obtain a well-understandable counterexample.

show abstract

Minimal Critical Subsystems for Discrete-Time Markov Models

Cited by 36 publications

References 17 publications

Permissive Supervisor Synthesis for Markov Decision Processes Through Learning

Permissive Supervisor Synthesis for Markov Decision Processes Through Learning

Safety-Aware Apprenticeship Learning

High-level Counterexamples for Probabilistic Automata

Contact Info

Product

Resources

About