Timely monitoring of partially observable stochastic systems

Sistla, A. Prasad; Žefran, Miloý; Feng, Yunli; Ben, Yue

doi:10.1145/2562059.2562136

Cited by 5 publications

(9 citation statements)

References 30 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We need the following technical lemma. The proof of this proposition employs ideas similar to those in [27] for proving an upper bound on the expected monitoring time for exponentially converging monitorable systems. Observe that as low decreases, the first term of the bound dominates.…”

Section: Monitors With One-sided Errormentioning

confidence: 99%

“…The alarm may trigger, e.g., a fail-safe way of shutting the system down. HMCs were suggested in [26,27] as models of partially observable stochastic systems. In this section, the monitor does not try to distinguish two HMCs, rather it tries to distinguish correct and faulty behavior of a single HMC.…”

Section: Application: Runtime Verifica-tionmentioning

confidence: 99%

“…Our work on exponentially decaying monitors is inspired by the exponentially converging monitorable systems defined in [27]. The algorithms presented there are for a very restricted class of HMCs, whereas our monitors work for all pairs of distinguishable HMCs.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Distinguishing Hidden Markov Chains

Kiefer

Sistla

2016

Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science

Self Cite

View full text Add to dashboard Cite

Hidden Markov Chains (HMCs) are commonly used mathematical models of probabilistic systems. They are employed in various fields such as speech recognition, signal processing, and biological sequence analysis. Motivated by applications in stochastic runtime verification, we consider the problem of distinguishing two given HMCs based on a single observation sequence that one of the HMCs generates. More precisely, given two HMCs and an observation sequence, a distinguishing algorithm is expected to identify the HMC that generates the observation sequence. Two HMCs are called distinguishable if for every ε > 0 there is a distinguishing algorithm whose error probability is less than ε. We show that one can decide in polynomial time whether two HMCs are distinguishable. Further, we present and analyze two distinguishing algorithms for distinguishable HMCs. The first algorithm makes a decision after processing a fixed number of observations, and it exhibits two-sided error. The second algorithm processes an unbounded number of observations, but the algorithm has only one-sided error. The error probability, for both algorithms, decays exponentially with the number of processed observations. We also provide an algorithm for distinguishing multiple HMCs.

show abstract

Section: Monitors With One-sided Errormentioning

confidence: 99%

Section: Application: Runtime Verifica-tionmentioning

confidence: 99%

See 1 more Smart Citation

Distinguishing Hidden Markov Chains

Kiefer

Sistla

2016

Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In previous work, real-time safety monitoring of CPS is based on the probability estimation that an execution is "bad" [8][9][10]. In our work, we are looking to develop decision rules that are able to improve on existing monitoring approaches.…”

Section: Real-time Monitoringmentioning

confidence: 99%

“…In this section we present the notions necessary for a development of a core theory behind a formal representation of CPS and a monitored property. Most of the definitions, unless explicitly indicated, are based on the work done in [8][9][10].…”

Section: Assembling Monitoring Of Cpsmentioning

confidence: 99%

Decision-Theoretic Monitoring of Cyber-Physical Systems

Yavolovsky

Žefran

Sistla

2016

Runtime Verification

Self Cite

View full text Add to dashboard Cite

Cyber-physical systems (CPS) represent "engineered systems that are built from, and depend upon, the seamless integration of computational algorithms and physical components".They can be found in such areas as aerospace, manufacturing, transportation, entertainment, healthcare, and automotive. For some of these systems the top priority is correct functioning; incorrect operation of systems like autonomous vehicles, medical devices or aircraft may lead to catastrophic consequences. But formally proving system correctness is a challenging problem. In recent years, runtime monitoring, where a monitor observes the outputs of the system and determines whether a system specification has been violated, has emerged as an attractive alternative.In this thesis, we focus on the decision-theoretic approach to monitoring of safety properties in CPS. In particular, we formulate the monitoring problem as a Partially Observable Markov Decision Process (POMDP) whereby deciding whether a run is safe corresponds to executing the optimal policy of the monitoring POMDP. We show how Monte-Carlo planning algorithm (POMCP) can be used to compute the optimal policy of the monitoring POMDP. The monitoring POMDP reward structure is naturally described with four parameters and an important question is how it affects the monitoring performance, quantified through acceptance accuracy, rejection accuracy and monitoring time. We analyze the performance of the POMDP-based monitors for special choices of the reward structure and compare them with the performance of the traditional threshold-based monitors. Our results show that using POMDPs we can somexii SUMMARY (Continued) times simultaneously improve both accuracies of the monitor while decreasing the monitoring time.Further, we study POMDP-based monitors for systems with terminal strongly connected components. For this class of systems, we derive the expressions for the POMDP value function.The expressions allow us to demonstrate how the POMDP-policy can take advantage of the properties of the monitored system and thus provide an enhanced monitoring performance.In order to study decision-theoretic monitors and experiment with them, we have developed a software called Decision-Theoretic Monitoring Tool (DTMT). This tool implements the architecture that allows easy integration of new monitoring decision rules and experimentation with an arbitrary user-defined system and property models.Finally, we evaluate POMDP monitors on an experimental robotic system. We simulate the operation of a simplified transmission system on a mobile robot that was developed specifically for this purpose. The experiments confirm that POMDP-based monitors provide an improvement over traditional threshold-based approaches. Further, we show that POMDPbased monitors implemented through POMCP can be used online even for large problems and thus provide an attractive and flexible alternative to traditional threshold-based monitors.xiii

show abstract