Statistical Model Checking

Abstract

We highlight the contributions made in the field of Statistical Model Checking (SMC) since its inception in 2002. As the formal setting, we use a very general model of Stochastic Systems (an SS is simply a family of time-indexed random variables), and Bounded LTL (BLTL) as the temporal logic. Let S be an SS and ϕ a BLTL formula. Our survey of the area is centered around the following five main contributions. Qualitative approach to SMC: Is the probability that S satisfies ϕ greater or equal to a certain threshold? Quantitative approach to SMC: What is the probability that S satisfies ϕ? Typically this results in a confidence interval being computed for this probability. Rare Events: What happens when the probability that S satisfies ϕ is extremely small, i.e. it is a rare event? To make the SMC approach viable in this setting, rare-event estimation techniques Importance Sampling and Importance Splitting are deployed to great advantage. Optimal Planning: Motivated by the success of Importance Sampling and Importance Splitting in rare-event SMC, we explore the use of these techniques in the context of optimal planning. In particular, we consider ARES, an optimal-planning approach based on a notion of adaptive receding-horizon planning. We illustrate the utility of ARES on the planning problem of bringing a flock of birds (autonomous agents) from a random initial configuration to a Vformation, an energy-conservation formation deployed by migrating geese. Somewhat ironically, the performance of ARES can be evaluated using (quantitative) SMC, as the problem to be solved is of the form F (J ≤ θ); i.e. does an ARES-generated plan eventually bring the flock to a configuration where the flock-wide cost function J is below a given threshold θ? Optimal Control: We show that the techniques we presented for optimal planning in the form of ARES carry over to the control setting in the form of Adaptive-Horizon Model-Predictive Control (AMPC). We again use the V-formation problem for evaluation purposes. We also introduce the concept of V-formation games, and show how the power of AMPC can be used to ward off cyber-physical attacks.

Language-Driven Engineering: From General-Purpose to Purpose-Specific Languages

“…Also the approaches presented in the remaining papers could profit from DSLs, e.g., as follows: [41] for specifying certain assertions or contracts, [43] for specifying data flow analyses 20 , [25] for specifying test models, [42] for defining learning alphabets or representing the learning result, [40] for modularly specifying the required code instrumentation, e.g. in an aspect-oriented fashion, and [15,19,66] for conveniently specifying their enriched system structures. Corresponding mIDEs (could) then guide the development by exploiting the DSL's specifics, e.g., the interpretation of assertions, security predicates, time, or probabilities.…”

“…On the other hand, LDE could also profit from the approaches presented in the other papers. In particular, all the involved analysis, verification and validation methods of [15,19,25,[40][41][42][43]61,66] are good candidates for inclusion in mIDEs in order to improve the development support and/or to control nonfunctional constraints. Finally, [27] provides a wealth of observations and techniques with potential to impact the future mIDE development.…”

Methods, Languages and Tools for Future System Development

“…Statistical Model Checking [23] is a verification technology for quantitative models of computer systems like the MDP discussed above. In contrast to classical verification, which provides yes/no-answers as a response to a verification task, statistical model checking answers probabilistically, indicating how well the considered system satisfies the property in question.…”

Runtime Verification Past Experiences and Future Projections

“…In other approaches, the result is a probability for a property to be satisfied, as in [77] (see discussion below). In statistical model checking [58], see also [60] in this volume, a stochastic system is executed multiple times, monitoring each execution against a temporal formula, computing either the probability that the system satisfies a formula (quantitative SMC), or determining whether the probability is greater than or equal to a certain treshold (qualitative SMC).…”