Learning continuous time Markov chains from sample executions

Sen, Koushik; Viswanathan, Mahesh; Agha, Gul

doi:10.1109/qest.2004.1348029

Cited by 30 publications

(45 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here the authors use the fact that one only needs to bound the type 1 errors (rejecting a correct compatibility hypothesis) for a number of tests that is linearly increasing, whereas the type 2 error (accepting an incorrect compatibility hypothesis) only needs to be bounded for a number of tests that depends on the (samplesize independent) number of states in the true model. Sen et al [13] only show that the probability of not learning the correct model can be made arbitrarily small, which is weaker than the probability one convergence of our Theorem 1.…”

Section: Methodsmentioning

confidence: 65%

“…A more complete argument is given in [13]. Here the authors use the fact that one only needs to bound the type 1 errors (rejecting a correct compatibility hypothesis) for a number of tests that is linearly increasing, whereas the type 2 error (accepting an incorrect compatibility hypothesis) only needs to be bounded for a number of tests that depends on the (samplesize independent) number of states in the true model.…”

Section: Methodsmentioning

confidence: 99%

“…In related work, Sen et al [13] previously also adapted the algorithm from [5] for learning Markov Chain models for verification. Our work differs from [13] in that we use a different modification of the original algorithm, which leads to somewhat stronger consistency results for learning in the limit, and that we analyse theoretically and experimentally how the convergence of the learned model relates to the convergence of probability estimates for system properties expressed in linear time temporal logic (LTL).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Probabilistic Automata for Model Checking

Mao

Chen

Jaeger

et al. 2011

2011 Eighth International Conference on Quantitative Evaluation of SysTems

View full text Add to dashboard Cite

Abstract-Obtaining accurate system models for verification is a hard and time consuming process, which is seen by industry as a hindrance to adopt otherwise powerful modeldriven development techniques and tools. In this paper we pursue an alternative approach where an accurate high-level model can be automatically constructed from observations of a given black-box embedded system. We adapt algorithms for learning finite probabilistic automata from observed system behaviors. We prove that in the limit of large sample sizes the learned model will be an accurate representation of the data-generating system. In particular, in the large sample limit, the learned model and the original system will define the same probabilities for linear temporal logic (LTL) properties. Thus, we can perform PLTL model-checking on the learned model to infer properties of the system. We perform experiments learning models from system observations at different levels of abstraction. The experimental results show the learned models provide very good approximations for relevant properties of the original system.

show abstract

Section: Methodsmentioning

confidence: 65%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Probabilistic Automata for Model Checking

Mao

Chen

Jaeger

et al. 2011

2011 Eighth International Conference on Quantitative Evaluation of SysTems

View full text Add to dashboard Cite

show abstract

“…Furthermore, our proposed approach is applicable to discrete stochastic systems. An interesting direction to investigate is its extension to continuous systems, such as continuous time Markov chains [23] or probabilistic timed automata.…”

Section: Resultsmentioning

confidence: 99%

Faster Statistical Model Checking by Means of Abstraction and Learning

Nouri

Raman

Bozga

et al. 2014

Runtime Verification

View full text Add to dashboard Cite

International audienceThis paper investigates the combined use of abstraction and probabilistic learning as a means to enhance statistical model checking performance. We are given a property (or a list of properties) for verification on a (large) stochastic system. We project on a set of traces generated from the original system, and learn a (small) abstract model from the projected traces, which contain only those labels that are relevant to the property to be verified. Then, we model-check the property on the reduced, abstract model instead of the large, original system. In this paper, we propose a formal definition of the projection on traces given a property to verify. We also provide conditions ensuring the correct preservation of the property on the abstract model. We validate our approach on the Herman's Self Stabilizing protocol. Our experimental results show that (a) the size of the abstract model and the verification time are drastically reduced, and that (b) the probability of satisfaction of the property being verified is correctly estimated by statistical model checking on the abstract model with respect to the concrete system

show abstract

“…For continuous-time Markov chains an identification method has been constructed (Sen et al 2004). This method is based on a method to infer probabilistic DFAs, known as ALER-GIA (Carrasco and Oncina 1994).…”

Section: Related Workmentioning

confidence: 99%

Efficiently identifying deterministic real-time automata from labeled data

2011

View full text Add to dashboard Cite

We develop a novel learning algorithm RTI for identifying a deterministic realtime automaton (DRTA) from labeled time-stamped event sequences. The RTI algorithm is based on the current state of the art in deterministic finite-state automaton (DFA) identification, called evidence-driven state-merging (EDSM). In addition to having a DFA structure, a DRTA contains time constraints between occurrences of consecutive events. Although this seems a small difference, we show that the problem of identifying a DRTA is much more difficult than the problem of identifying a DFA: identifying only the time constraints of a DRTA given its DFA structure is already NP-complete. In spite of this additional complexity, we show that RTI is a correct and complete algorithm that converges efficiently (from polynomial time and data) to the correct DRTA in the limit. To the best of our knowledge, this is the first algorithm that can identify a timed automaton model from time-stamped event sequences.A straightforward alternative to identifying DRTAs is to identify a DFA that models time implicitly, i.e., a DFA that uses different states for different points in time. Such a DFA can be identified by first sampling the timed sequences using a fixed frequency, and subsequently applying EDSM to the resulting non-timed event sequences. We evaluate the performance of both RTI and this sampling approach experimentally on artificially generated data. In these Mach Learn (2012) 86:295-333 experiments RTI outperforms the sampling approach significantly. Thus, we show that if we obtain data from a real-time system, it is easier to identify a DRTA from this data than to identify an equivalent DFA.

show abstract

Learning continuous time Markov chains from sample executions

Cited by 30 publications

References 15 publications

Learning Probabilistic Automata for Model Checking

Learning Probabilistic Automata for Model Checking

Faster Statistical Model Checking by Means of Abstraction and Learning

Efficiently identifying deterministic real-time automata from labeled data

Contact Info

Product

Resources

About