Partial Order Reduction for Markov Decision Processes: A Survey

Groesser, Marcus; Baier, Christel

doi:10.1007/11804192_19

“…As a result, symbolic model checkers can run out of memory while verifying a typical CTMC model with as few as 73,000 states [23]. State reduction techniques, such as bisimulation minimization [7,8,14], abstraction [6,12,14,20], symmetry reduction [5,16], and partial order reduction [9] have been mainly extended to discrete-time, finite-state probabilistic systems. The three-valued abstraction [14] can reduce large, finite-state CTMCs.…”

Section: Introductionmentioning

confidence: 99%

STAMINA: STochastic Approximate Model-Checker for INfinite-State Analysis

Neupane

¹

,

Myers

²

,

Madsen

³

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Stochastic model checking is a technique for analyzing systems that possess probabilistic characteristics. However, its scalability is limited as probabilistic models of real-world applications typically have very large or infinite state space. This paper presents a new infinite state CTMC model checker, STAMINA, with improved scalability. It uses a novel state space approximation method to reduce large and possibly infinite state CTMC models to finite state representations that are amenable to existing stochastic model checkers. It is integrated with a new property-guided state expansion approach that improves the analysis accuracy. Demonstration of the tool on several benchmark examples shows promising results in terms of analysis efficiency and accuracy compared with a state-of-theart CTMC model checker that deploys a similar approximation method.

show abstract

“…As a result, symbolic model checkers can run out of memory while verifying a typical CTMC model with as few as 73,000 states [24]. State reduction techniques, such as bisimulation minimization [15,7,8], abstraction [21,15,6,13], symmetry reduction [17,5], and partial order reduction [9] have been mainly extended to discrete-time, finite-state probabilistic systems. The three-valued abstraction [15] can reduce large, finite-state CTMCs.…”

Section: Introductionmentioning

confidence: 99%

STAMINA: STochastic Approximate Model-checker for INfinite-state Analysis

Neupane

¹

,

Myers

²

,

Madsen

³

et al. 2019

Preprint

0

View full text Add to dashboard Cite

Stochastic model checking is a technique for analyzing systems that possess probabilistic characteristics. However, its scalability is limited as probabilistic models of real-world applications typically have very large or infinite state space. This paper presents a new infinite state CTMC model checker, STAMINA, with improved scalability. It uses a novel state space approximation method to reduce large and possibly infinite state CTMC models to finite state representations that are amenable to existing stochastic model checkers. It is integrated with a new property-guided state expansion approach that improves the analysis accuracy. Demonstration of the tool on several benchmark examples shows promising results in terms of analysis efficiency and accuracy compared with a state-of-theart CTMC model checker that deploys a similar approximation method.

show abstract

“…Related work. Abstraction-refinement has been applied to reachability problems in MDPs [10], partial-order reduction techniques using Peled's ample-set method have been generalised to MDPs [13], abstract interpretation has been applied to MDPs [20], and various bisimulation equivalences and simulation pre-orders allow model aggregation prior to model checking, see e. g., [4,23]. Recent techniques that have been proposed include abstraction of MDPs by two-player stochastic games [18], and symmetry reduction [19].…”

Section: Introductionmentioning

confidence: 99%

Three-Valued Abstraction for Continuous-Time Markov Chains

Katoen

¹

,

Klink

²

,

Leucker³

et al.

Computer Aided Verification

View full text Add to dashboard Cite

This paper proposes a novel abstraction technique for continuous-time Markov chains (CTMCs). Our technique fits within the realm of three-valued abstraction methods that have been used successfully for traditional model checking. The key idea is to apply abstraction on uniform CTMCs that are readily obtained from general CTMCs, and to abstract transition probabilities by intervals. It is shown that this provides a conservative abstraction for both true and false for a threevalued semantics of the branching-time logic CSL (Continuous Stochastic Logic). Experiments on an infinite-state CTMC indicate the feasibility of our abstraction technique.

show abstract

Partial Order Reduction for Markov Decision Processes: A Survey

Cited by 13 publications

References 31 publications

STAMINA: STochastic Approximate Model-Checker for INfinite-State Analysis

STAMINA: STochastic Approximate Model-Checker for INfinite-State Analysis

STAMINA: STochastic Approximate Model-checker for INfinite-state Analysis

Three-Valued Abstraction for Continuous-Time Markov Chains

Contact Info

Product

Resources

About