Ergodic Control of Switching Diffusions

Ghosh, Mrinal K.; Arapostathis, Ari; Marcus, Solomon

doi:10.1137/s0363012996299302

Cited by 207 publications

(125 citation statements)

References 18 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…We use the notion of stochastic kernel as a unifying concept for our particular model of stochastic hybrid system. Note that, unlike other models of stochastic hybrid systems [18,6,8,35, 37], we do not consider continuous stochastic transitions, since those are less relevant for SF/SL applications.…”

Section: Modelmentioning

confidence: 99%

Bayesian Statistical Model Checking with Application to Stateflow/Simulink Verification

Zuliani¹,

Platzer²,

Clarke³

2010

144

View full text Add to dashboard Cite

We address the problem of model checking stochastic systems, i.e., checking whether a stochastic system satisfies a certain temporal property with a probability greater (or smaller) than a fixed threshold. In particular, we present a Statistical Model Checking (SMC) approach based on Bayesian statistics. We show that our approach is feasible for a certain class of hybrid systems with stochastic transitions, a generalization of Simulink/Stateflow models. Standard approaches to stochastic discrete systems require numerical solutions for large optimization problems and quickly become infeasible with larger state spaces. Generalizations of these techniques to hybrid systems with stochastic effects are even more challenging. The SMC approach was pioneered by Younes and Simmons in the discrete and non-Bayesian case. It solves the verification problem by combining randomized sampling of system traces (which is very efficient for Simulink/Stateflow) with hypothesis testing (i.e., testing against a probability threshold) or estimation (i.e., computing with high probability a value close to the true probability). We believe SMC is essential for scaling up to large Stateflow/Simulink models. While the answer to the verification problem is not guaranteed to be correct, we prove that Bayesian SMC can make the probability of giving a wrong answer arbitrarily small. The advantage is that answers can usually be obtained much faster than with standard, exhaustive model checking techniques. We apply our Bayesian SMC approach to a representative example of stochastic discrete-time hybrid system models in Stateflow/Simulink: a fuel control system featuring hybrid behavior and fault tolerance. We show that our technique enables faster verification than state-of-the-art statistical techniques. We emphasize that Bayesian SMC is by no means restricted to Stateflow/Simulink models. It is in principle applicable to a variety of stochastic models from other domains, e.g., systems biology.

show abstract

Section: Modelmentioning

confidence: 99%

Bayesian Statistical Model Checking with Application to Stateflow/Simulink Verification

Zuliani¹,

Platzer²,

Clarke³

2010

144

View full text Add to dashboard Cite

show abstract

“…The work of Filar et al [17] is a notable exception but requires a time-scale separation between the (purely deterministic) continuous dynamics and the discrete jump dynamics. In switched diffusion processes (SDPs), as defined by Ghosh et al [18], the evolution of the continuous state in each mode is modeled by a stochastic differential equation and transitions between modes are controlled by a continuous-time Markov process. The transition rates of the Markov process can depend on the state but transitions do not generate jumps on the continuous state (i.e., no resets).…”

Section: Introductionmentioning

confidence: 99%

“…The transition rates of the Markov process can depend on the state but transitions do not generate jumps on the continuous state (i.e., no resets). The reader is referred to [42] for a comparison of the models in [14,18,21]. The SHSs considered here can be viewed as special cases of general jump-diffusion processes [24].…”

Section: Introductionmentioning

confidence: 99%

Stochastic Hybrid Systems: Application to Communication Networks

Hespanha

2004

Hybrid Systems: Computation and Control

129

122

View full text Add to dashboard Cite

Abstract. We propose a model for Stochastic Hybrid Systems (SHSs) where transitions between discrete modes are triggered by stochastic events much like transitions between states of a continuous-time Markov chains. However, the rate at which transitions occur is allowed to depend both on the continuous and the discrete states of the SHS. Based on results available for Piecewise-Deterministic Markov Process (PDPs), we provide a formula for the extended generator of the SHS, which can be used to compute expectations and the overall distribution of the state. As an application, we construct a stochastic model for on-off TCP flows that considers both the congestion-avoidance and slow-start modes and takes directly into account the distribution of the number of bytes transmitted. Using the tools derived for SHSs, we model the dynamics of the moments of the sending rate by an infinite system of ODEs, which can be truncated to obtain an approximate finite-dimensional model. This model shows that, for transfer-size distributions reported in the literature, the standard deviation of the sending rate is much larger than its average. Moreover, the later seems to vary little with the probability of packet drop. This has significant implications for the design of congestion control mechanisms.

show abstract

“…Here, we consider a restricted form of SHSs that are closely related to (and heavily inspired by) the PiecewiseDeterministic Markov Process (PDMPs) introduced by Davis [11] and, in fact, our SHSs can be viewed as a special case of PDMPs and thus inherent many of the PDMPs properties. SHSs are also closely related to Markov Jump Linear Systems (MJLS) [10,20] and to Switching Difusions (SD) [13,27], which differ from our SHSs in that the emphasis in MJLSs and SDs is in the change in dynamics at a set of event times and not on the impulsive effects that are fundamental in many NCSs. Nevertheless, MJLSs have been successfully used to study fairly complex NCSs [8].…”

Section: Stochastic Hybrid Systemsmentioning

confidence: 99%

Networked Control Systems

Wang

Liu

2008

200

View full text Add to dashboard Cite

This paper aims at familiarizing the reader with Stochastic Hybrid Systems (SHSs) and enabling her to use these systems to model and analyze Networked Control Systems (NCSs). Towards this goal, we introduce two different models of SHSs and a set of theoretical tools for their analysis. In parallel with the presentation of the mathematical models and results, we provide a few simple examples that illustrate the use of SHSs to models NCSs.Keywords: Network Control Systems; Hybrid Systems; Stochastic Processes; Stability; Markov Processes Networked Control SystemsThe expression Networked Control Systems (NCSs) typically refers to feedback control systems for which some of the sensors, controllers, and actuators communicate with each other using a shared communication network. The use of a multi-purpose shared network reduces installation and maintenance costs and adds flexibility, as it permits the system reconfiguration and/or expansion with minimal additional infrastructure costs. In view of this, NCSs are finding application in numerous areas that include the automotive industry, the aviation industry, robotics, process control, and building control, among others.While NCSs are attractive from the perspective of cost of deployment and maintenance, they introduce significant design challenges, because the traditional unity feedback loop that operates in continuous time or at a fixed sampling rate is not adequate when sensor data arrives from multiple sources, asynchronously, delayed, and possibly corrupted. Consequently, NCSs have been the focus of intense study in the last few years [16,25,29]. This paper is focused on two aspects of NCSs that are responsible for important challenges in analyzing and designing NCSs and that are prompting the development of new formal tools to study these systems. This paper is partially based on material presented in a plenary lecture at the 4th IFAC Workshop on Distributed Estimation and Control in Networked Systems (NECSYS'13).

show abstract

Ergodic Control of Switching Diffusions

Cited by 207 publications

References 18 publications

Bayesian Statistical Model Checking with Application to Stateflow/Simulink Verification

Bayesian Statistical Model Checking with Application to Stateflow/Simulink Verification

Stochastic Hybrid Systems: Application to Communication Networks

Networked Control Systems

Contact Info

Product

Resources

About