Predictive Run-Time Verification of Discrete-Time Reachability Properties in Black-Box Systems Using Trace-Level Abstraction and Statistical Learning

Babaee, Reza; Gurfinkel, Arie; Fischmeister, Sebastian

doi:10.1007/978-3-030-03769-7_11

Cited by 16 publications

(14 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the properties of convex sets [18, Ch. 2], we make the following observations: If B is convex, est up MDP (B, z) is convex, as all operations in computing a new belief are convexset preserving 7 . Furthermore, if B has a finite set of vertices, then est up MDP (B, z) has a finite set of vertices.…”

Section: Properties Of Est Mdp (τ )mentioning

confidence: 99%

Runtime Monitors for Markov Decision Processes

Junges

Torfah

Seshia

2021

Computer Aided Verification

View full text Add to dashboard Cite

We investigate the problem of monitoring partially observable systems with nondeterministic and probabilistic dynamics. In such systems, every state may be associated with a risk, e.g., the probability of an imminent crash. During runtime, we obtain partial information about the system state in form of observations. The monitor uses this information to estimate the risk of the (unobservable) current system state. Our results are threefold. First, we show that extensions of state estimation approaches do not scale due the combination of nondeterminism and probabilities. While exploiting a geometric interpretation of the state estimates improves the practical runtime, this cannot prevent an exponential memory blowup. Second, we present a tractable algorithm based on model checking conditional reachability probabilities. Third, we provide prototypical implementations and manifest the applicability of our algorithms to a range of benchmarks. The results highlight the possibilities and boundaries of our novel algorithms.

show abstract

Section: Properties Of Est Mdp (τ )mentioning

confidence: 99%

Runtime Monitors for Markov Decision Processes

Junges

Torfah

Seshia

2021

Computer Aided Verification

View full text Add to dashboard Cite

show abstract

“…The work of [3] addresses the predictive monitoring problem for stochastic black-box systems, where a Markov model is inferred offline from observed traces and used to construct a predictive runtime monitor for probabilistic reachability checking. In contrast to NSC, this method focuses on discrete-space models, which allows the predictor to be represented as a look-up table (as opposed to a neural network).…”

Section: Related Workmentioning

confidence: 99%

Neural Predictive Monitoring

Bortolussi

Cairoli

Paoletti

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Neural State Classification (NSC) is a recently proposed method for runtime predictive monitoring of Hybrid Automata (HA) using deep neural networks (DNNs). NSC trains a DNN as an approximate reachability predictor that labels a given HA state x as positive if an unsafe state is reachable from x within a given time bound, and labels x as negative otherwise. NSC predictors have very high accuracy, yet are prone to prediction errors that can negatively impact reliability. To overcome this limitation, we present Neural Predictive Monitoring (NPM), a technique based on NSC and conformal prediction that complements NSC predictions with statistically sound estimates of uncertainty. This yields principled criteria for the rejection of predictions likely to be incorrect, without knowing the true reachability values. We also present an active learning method that significantly reduces both the NSC predictor's error rate and the percentage of rejected predictions. Our approach is highly efficient, with computation times on the order of milliseconds, and effective, managing in our experimental evaluation to successfully reject almost all incorrect predictions.

show abstract

“…The work of [47,48] addresses the predictive monitoring problem for stochastic black-box systems, where a Markov model is inferred offline from observed traces and used to construct a predictive runtime monitor for probabilistic reachability checking. In contrast to NSC, this method focuses on discrete-space models, which allows the predictor to be represented as a look-up table, as opposed to a neural network.…”

Section: Related Workmentioning

confidence: 99%

Neural predictive monitoring and a comparison of frequentist and Bayesian approaches

Bortolussi

Cairoli

Paoletti

et al. 2021

Int J Softw Tools Technol Transfer

View full text Add to dashboard Cite

Neural state classification (NSC) is a recently proposed method for runtime predictive monitoring of hybrid automata (HA) using deep neural networks (DNNs). NSC trains a DNN as an approximate reachability predictor that labels an HA state x as positive if an unsafe state is reachable from x within a given time bound, and labels x as negative otherwise. NSC predictors have very high accuracy, yet are prone to prediction errors that can negatively impact reliability. To overcome this limitation, we present neural predictive monitoring (NPM), a technique that complements NSC predictions with estimates of the predictive uncertainty. These measures yield principled criteria for the rejection of predictions likely to be incorrect, without knowing the true reachability values. We also present an active learning method that significantly reduces the NSC predictor’s error rate and the percentage of rejected predictions. We develop two versions of NPM based, respectively, on the use of frequentist and Bayesian techniques to learn the predictor and the rejection rule. Both versions are highly efficient, with computation times on the order of milliseconds, and effective, managing in our experimental evaluation to successfully reject almost all incorrect predictions. In our experiments on a benchmark suite of six hybrid systems, we found that the frequentist approach consistently outperforms the Bayesian one. We also observed that the Bayesian approach is less practical, requiring a careful and problem-specific choice of hyperparameters.

show abstract

Predictive Run-Time Verification of Discrete-Time Reachability Properties in Black-Box Systems Using Trace-Level Abstraction and Statistical Learning

Cited by 16 publications

References 35 publications

Runtime Monitors for Markov Decision Processes

Runtime Monitors for Markov Decision Processes

Neural Predictive Monitoring

Neural predictive monitoring and a comparison of frequentist and Bayesian approaches

Contact Info

Product

Resources

About