A Statistical Model Checker for Nondeterminism and Rare Events

Budde, Carlos E.; D’Argenio, Pedro R.; Hartmanns, Arnd; Sedwards, Sean

doi:10.1007/978-3-319-89963-3_20

Cited by 33 publications

(45 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…EXP selects thresholds and an individual splitting factor for each threshold, removing the need for the user to manually select a global splitting factor. We implemented all techniques in the Fig tool [11] and the modes simulator [15] of the Modest Toolset [34]. The techniques can be freely combined, and work for all the formalisms supported by the two tools-including CTMC, input-output stochastic automata (IOSA [22]), and stochastic timed automata (STA [9]).…”

Section: Contributionsmentioning

confidence: 99%

“…We implemented the compositional importance function generation of Section 3, the splitting methods described in Section 4, and the threshold calculation methods of Section 5 in the modes simulator [15] of the Modest Toolset [31] and the Fig tool [11] for input-output stochastic automata.…”

Section: Tools Languages and Modelsmentioning

confidence: 99%

“…Importance splitting enjoys a rich scope of applications, most prominently in queueing theory but also in e.g. dependability analysis, randomised algorithms, distributed systems, particle transport, and hybrid systems [8,15,42,51].Thus, to tune a particular implementation, both methods require certain knowledge of the specific setting where they are applied. Overall, importance splitting appears more amenable to automatic approaches across different modelling formalisms using different kinds of probability distributions due to its black-box view of the model.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Automated compositional importance splitting

Budde¹,

D’Argenio²,

Hartmanns³

2019

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

In the formal verification of stochastic systems, statistical model checking uses simulation to overcome the state space explosion problem of probabilistic model checking. Yet its runtime explodes when faced with rare events, unless a rare event simulation method like importance splitting is used. The effectiveness of importance splitting hinges on nontrivial model-specific inputs: an importance function with matching splitting thresholds. This prevents its use by nonexperts for general classes of models. In this paper, we present an automated method to derive the importance function. It considers both the structure of the model and of the formula characterising the rare event. It is memory-efficient by exploiting the compositional nature of formal models. We experimentally evaluate it in various combinations with two approaches to threshold selection as well as different splitting techniques for steady-state and transient properties. We find that Restart splitting combined with thresholds determined via a new expected success method most reliably succeeds and performs very well for transient properties. It remains competitive in the steady-state case, which is however challenging to all combinations we consider. All methods are implemented in the modes tool of the Modest Toolset and in the Fig rare event simulator.state space explosion problem limits this approach to small models. For other models, in particular those involving events governed by general continuous probability distributions, model checking techniques only exist for specific subclasses with limited scalability [55] or merely compute probability bounds [31].Statistical model checking (SMC [38,72]), i.e. using Monte Carlo simulation with formal models, has become a popular alternative for large models, and for formalisms not amenable to (traditional) probabilistic model checking like stochastic (timed) automata [9,18]. SMC trades memory for runtime: memory usage is constant, but the number of simulation runs which are needed to converge to a result can easily explode with the desired precision. This is exacerbated in the presence of rare events. For instance, when the true probability of an event is 10 −15 , one may want that the error of an estimation is no larger than 10 −16 . Such tight requirements in the precision of estimations may render traditional Monte Carlo simulation approaches infeasible [24,65].Rare event simulation methods (RES [58]) have been developed to attack this problem. They increase the number of simulation runs that reach the rare event and adjust the statistical evaluation accordingly. Broadly speaking, the main RES methods are importance sampling and importance splitting. They complement each other in several application domains [57]. The former modifies the probability distributions which dictate the stochastic behaviour of the model, with the aim to make the event more likely to occur. The challenge lies in finding a "good" change of measure to modify probabilities in an effective way. Importance splitting instead does n...

show abstract

Section: Contributionsmentioning

confidence: 99%

Section: Tools Languages and Modelsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Automated compositional importance splitting

Budde¹,

D’Argenio²,

Hartmanns³

2019

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the experiments on the Heated Tank benchmark, we have used the Modest Toolset's simulator "modes" [8] and its support for rare event simulation based on importance splitting with the fixed effort method (using 64 child runs for each fixed effort run) [7].…”

Section: Toolsmentioning

confidence: 99%

ARCH-COMP19 Category Report: Stochastic Modelling

Abate

Blom

Cauchi

et al.

EPiC Series in Computing

View full text Add to dashboard Cite

This report presents the results of a friendly competition for formal verification and policy synthesis of stochastic models. It also introduces new benchmarks within this category, and recommends next steps for this category towards next year's edition of the competition. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in Spring 2019.

show abstract

Lightweight Statistical Model Checking in Nondeterministic Continuous Time

D’Argenio

Hartmanns

Sedwards

2018

Leveraging Applications of Formal Methods, Verification and Validation. Verification

Self Cite

View full text Add to dashboard Cite

Lightweight scheduler sampling brings statistical model checking to nondeterministic formalisms with undiscounted properties, in constant memory. Its direct application to continuous-time models is rendered ineffective by their dense concrete state spaces and the need to consider continuous input for optimal decisions. In this paper we describe the challenges and state of the art in applying lightweight scheduler sampling to three continuous-time formalisms: After a review of recent work on exploiting discrete abstractions for probabilistic timed automata, we discuss scheduler sampling for Markov automata and apply it on two case studies. We provide further insights into the tradeoffs between scheduler classes for stochastic automata. Throughout, we present extended experiments and new visualisations of the distribution of schedulers.

show abstract

A Statistical Model Checker for Nondeterminism and Rare Events

Cited by 33 publications

References 45 publications

Automated compositional importance splitting

Automated compositional importance splitting

ARCH-COMP19 Category Report: Stochastic Modelling

Lightweight Statistical Model Checking in Nondeterministic Continuous Time

Contact Info

Product

Resources

About