Learning deterministic probabilistic automata from a model checking perspective

Abstract

. (2016). Learning deterministic probabilistic automata from a model checking perspective. Machine Learning, 105(2), 255-299. https://doi.org/10.1007/s10994-016-5565-9General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.? Users may download and print one copy of any publication from the public portal for the purpose of private study or research. ? You may not further distribute the material or use it for any profit-making activity or commercial gain ? You may freely distribute the URL identifying the publication in the public portal ? Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access to the work immediately and investigate your claim. Abstract Probabilistic automata models play an important role in the formal design and analysis of hard-and software systems. In this area of applications, one is often interested in formal model-checking procedures for verifying critical system properties. Since adequate system models are often difficult to design manually, we are interested in learning models from observed system behaviors. To this end we adopt techniques for learning finite probabilistic automata, notably the Alergia algorithm. In this paper we show how to extend the basic algorithm to also learn automata models for both reactive and timed systems. A key question of our investigation is to what extent one can expect a learned model to be a good approximation for the kind of probabilistic properties one wants to verify by model checking. We establish theoretical convergence properties for the learning algorithm as well as for probability estimates of system properties expressed in linear time temporal logic and linear continuous stochastic logic. We empirically compare the learning algorithm with statistical model checking and demonstrate the feasibility of the approach for practical system verification.

$$L^*$$-Based Learning of Markov Decision Processes

“…Measurement Setup. As in [30], we configure IoAlergia with a data-dependent significance parameter for the compatibility check, by setting ǫ N = 10000 N , where N is the total combined length of all traces used for learning. This parameter serves a role analogous to the α parameter for the Hoeffding bounds used by L * mdp .…”

“…In contrast to IoAlergia, we observed that L * mdp shows better performance with non-data-dependent α, therefore we set α = 0.05 for all experiments. Motivated by convergence guarantees given in [30], we collect traces for IoAlergia by sampling with a scheduler that selects inputs according to a uniform distribution. The length of these traces is geometrically distributed with a parameter p l and the number of traces is chosen such that IoAlergia and L * mdp learn from approximately the same amount of data.…”

“…An MDP is deterministic if ∀q ∈ Q, ∀i : δ(q, i)(q ′ ) > 0 ∧ δ(q, i)(q ′′ ) > 0 → q ′ = q ′′ ∨ L(q ′ ) = L(q ′′ ). We learn deterministic labelled MDPs as learned by passive learning techniques like IoAlergia [30]. Such MDPs define at most one successor state for each source state and input-output pair.…”

“…9. More concretely, let α n = 1 n r for r > 2 as used by Mao et al [30], which implies n α n n < ∞. For the remainder of this section, let S n , E n , T n be the closed and consistent observation table containing the first n samples stored by the teacher in the multiset S n .…”

“…We refer to the former as exact learning algorithm L * mdp e and to the latter as sampling-based learning algorithm L * mdp . We implemented L * mdp and evaluated it by comparing it to IoAlergia [29,30]. Experiments showed favourable performance of L * mdp , i.e.…”