Active Automata Learning in Practice

Howar, Falk; Steffen, Bernhard

doi:10.1007/978-3-319-96562-8_5

Cited by 56 publications

(32 citation statements)

References 130 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1), BBC can guarantee that the given black-box system certainly satisfies the given property [44] although the soundness relies on additional assumptions on the black-box system (e.g., the upper bound of the number of the states). A recent survey [25] reports that at the early stage of the automata learning, it is beneficial for the equivalence testing to try to find a counterexample ι ∈ Σ * satisfying M(ι) M (ι) instead of trying to prove the equivalence by conformance testing, e.g., W-method [11] and Wp-method [23]. Random testing is one typical choice of the equivalence testing other than conformance testing.…”

Section: The Falsification Problemmentioning

confidence: 99%

See 1 more Smart Citation

Falsification of cyber-physical systems with robustness-guided black-box checking

Waga

2020

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

View full text Add to dashboard Cite

For exhaustive formal verification, industrial-scale cyberphysical systems (CPSs) are often too large and complex, and lightweight alternatives (e.g., monitoring and testing) have attracted the attention of both industrial practitioners and academic researchers. Falsification is one popular testing method of CPSs utilizing stochastic optimization. In stateof-the-art falsification methods, the result of the previous falsification trials is discarded, and we always try to falsify without any prior knowledge. To concisely memorize such prior information on the CPS model and exploit it, we employ Black-box checking (BBC), which is a combination of automata learning and model checking. Moreover, we enhance BBC using the robust semantics of STL formulas, which is the essential gadget in falsification. Our experiment results suggest that our robustness-guided BBC outperforms a state-of-the-art falsification tool. CCS CONCEPTS• Computer systems organization → Embedded and cyber-physical systems; Real-time system specification; • Software and its engineering → Formal software verification; Search-based software engineering.

show abstract

Section: The Falsification Problemmentioning

confidence: 99%

“…These benchmarks are mainly used to compare the scalability with respect to the size of the benchmark. (5,20), (5,25), (15,30), 26] (v < p 1 ) ∨ □ [28,28] (v > p 2 )…”

Section: Experimental Evaluationmentioning

confidence: 99%

Falsification of cyber-physical systems with robustness-guided black-box checking

Waga

2020

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

View full text Add to dashboard Cite

show abstract

“…In [55], automata learning is used to learn embedded software programs of printers. Though such research indicates the use of active automata learning for real-life systems, challenges exist to broaden its impact for practical use [24,42,43]. There are very limited examples on the use of active automata learning in an automotive context [34,54] and it is yet to find its place in automotive software development.…”

Section: Related Workmentioning

confidence: 99%

“…Active automata learning [1,5,11,16,19,24,28,56] is an active field of research that addresses the problem of automatic model construction. These approaches constitute a class of machine learning algorithms that aim to deduce a finite-state automaton describing the behaviour of a system by actively interacting with the target system.…”

Section: Introductionmentioning

confidence: 99%

Automatically learning formal models

Selvaraj

Farooqui

Panahandeh

et al. 2020

Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedi

View full text Add to dashboard Cite

The correctness of autonomous driving software is of utmost importance as incorrect behaviour may have catastrophic consequences. Though formal model-based engineering techniques can help guarantee correctness, challenges exist in widespread industrial adoption. One among them is the model construction problem. Manual construction of formal models is expensive, error-prone, and intractable for large systems. Automating model construction would be a great enabler for the use of formal methods to guarantee software correctness and thereby for safe deployment of autonomous vehicles. Such automated techniques can be beneficial in software design, re-engineering, and reverse engineering. In this industrial case study, we apply active learning techniques to obtain formal models from an existing autonomous driving software (in development) implemented in MATLAB. We demonstrate the feasibility of active automata learning algorithms for automotive industrial use. Furthermore, we discuss the practical challenges in applying automata learning and possible directions for integrating automata learning into automotive software development workflow. CCS CONCEPTS • Software and its engineering → Software design engineering; Software development methods; Formal software verification; Software reverse engineering; • Computing methodologies → Model development and analysis; Machine learning; • Theory of computation → Models of computation; Formal languages and automata theory.

show abstract

“…Equivalence Query. Equivalence queries are often implemented via (conformance) testing in active automata [23], e.g., via the W-method [16] method for deterministic models. Such testing techniques generally execute some test suite to find counterexamples to conformance between a model and the SUL.…”

Section: Teacher Implementationmentioning

confidence: 99%

$$L^*$$-Based Learning of Markov Decision Processes

Tappler

Aichernig

Bacci

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Automata learning techniques automatically generate system models from test observations. These techniques usually fall into two categories: passive and active. Passive learning uses a predetermined data set, e.g., system logs. In contrast, active learning actively queries the system under learning, which is considered more efficient. An influential active learning technique is Angluin's L * algorithm for regular languages which inspired several generalisations from DFAs to other automata-based modelling formalisms. In this work, we study L *based learning of deterministic Markov decision processes, first assuming an ideal setting with perfect information. Then, we relax this assumption and present a novel learning algorithm that collects information by sampling system traces via testing. Experiments with the implementation of our sampling-based algorithm suggest that it achieves better accuracy than state-of-the-art passive learning techniques with the same amount of test data. Unlike existing learning algorithms with predefined states, our algorithm learns the complete model structure including the states.

show abstract

Active Automata Learning in Practice

Cited by 56 publications

References 130 publications

Falsification of cyber-physical systems with robustness-guided black-box checking

Falsification of cyber-physical systems with robustness-guided black-box checking

Automatically learning formal models

$$L^*$$-Based Learning of Markov Decision Processes

Contact Info

Product

Resources

About