Stochastic Hybrid Systems: Application to Communication Networks

Hespanha, João P.

doi:10.1007/978-3-540-24743-2_26

Cited by 130 publications

(125 citation statements)

References 40 publications

(52 reference statements)

Supporting

Mentioning

122

Contrasting

Unclassified

Order By: Relevance

“…We model the stochastic logistic model as a Stochastic Hybrid System (SHS) whose state x is the population size of the species. Then, the time evolution of the moments is obtained using results from the SHS literature (e.g., Hespanha, 2004). Details of the stochastic logistic model and its modeling as a SHS are presented in Section 2.…”

Section: Introductionmentioning

confidence: 99%

A Derivative Matching Approach to Moment Closure for the Stochastic Logistic Model

Singh

Hespanha

2007

Bull. Math. Biol.

View full text Add to dashboard Cite

Continuous-time birth-death Markov processes serve as useful models in population biology. When the birth-death rates are nonlinear, the time evolution of the first n order moments of the population is not closed, in the sense that it depends on moments of degree higher than n. For analysis purpose, the time evolution of the first n order moments is often made to be closed by approximating these higher order moments as a nonlinear function of moments up to order n, which we refer to as the moment closure function.In this paper, a systematic procedure for constructing moment closure functions of arbitrary order is presented for the stochastic logistic model. We obtain the moment closure function by first assuming a certain separable form for it, and then matching time derivatives of the exact (not closed) moment equations with that of the approximate (closed) equations for some initial time and set of initial conditions. The separable structure ensures that the steady-state solutions for the approximate equations are unique, real and positive, while the derivative matching guarantees a good approximation, at-least locally in time. Explicit formulas to construct these moment closure functions for arbitrary order of truncation n are provided with higher values of n leading to better approximation of the actual moment dynamics.A host of other moment closure functions previously proposed in the literature are also investigated. Among these we show that only the ones that achieve derivative matching provide a close approximation to the exact solution. Moreover, we improve the accuracy of several previously proposed moment closure functions by forcing derivative matching.

show abstract

Section: Introductionmentioning

confidence: 99%

A Derivative Matching Approach to Moment Closure for the Stochastic Logistic Model

Singh

Hespanha

2007

Bull. Math. Biol.

View full text Add to dashboard Cite

show abstract

“…In essence, between transition counter increments the discrete state remains constant whereas the continuous state flows according to (1). At transition times, the continuous and discrete states are reset according to (2).…”

Section: Polynomial Stochastic Hybrid Systemsmentioning

confidence: 99%

“…The model used here for SHSs, whose formal definition can be found in Sec. 2, was introduced in [1] and is heavily inspired by the Piecewise-Deterministic Markov Processes (PDPs) in [2]. Alternative models can be found in [3,4,5].…”

Section: Introductionmentioning

confidence: 99%

“…The extended generator of a stochastic hybrid system allows one to compute the time-derivative of a "test function" of the state of the SHS along solutions to the system, and can be viewed as a generalization of the Lie derivative for deterministic systems [1,2]. Polynomial stochastic hybrid systems (pSHSs) are characterized by extended generators that map polynomial test functions into polynomials.…”

Section: Introductionmentioning

confidence: 99%

“…We consider two distinct cases: long-lived traffic corresponding to the transfer of files with infinite length; and on-off traffic consisting of file transfers with exponentially distributed lengths, alternated by times of inactivity (also exponentially distributed). These examples are motivated by [1,6]. 2.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Polynomial Stochastic Hybrid Systems

Hespanha

2005

Hybrid Systems: Computation and Control

View full text Add to dashboard Cite

Abstract. This paper deals with polynomial stochastic hybrid systems (pSHSs), which generally correspond to stochastic hybrid systems with polynomial continuous vector fields, reset maps, and transition intensities. For pSHSs, the dynamics of the statistical moments of the continuous states evolve according to infinite-dimensional linear ordinary differential equations (ODEs). We show that these ODEs can be approximated by finite-dimensional nonlinear ODEs with arbitrary precision. Based on this result, we provide a procedure to build this type of approximations for certain classes of pSHSs. We apply this procedure for several examples of pSHSs and evaluate the accuracy of the results obtained through comparisons with Monte Carlo simulations. These examples include: the modeling of TCP congestion control both for long-lived and on-off flows; state-estimation for networked control systems; and the stochastic modeling of chemical reactions.

show abstract

Partially‐observable stochastic hybrid systems (poshss) state estimation and optimal control

Liu¹,

Hwang²

2015

Asian Journal of Control

View full text Add to dashboard Cite

This paper discusses the state estimation and optimal control problem of a class of partially‐observable stochastic hybrid systems (POSHS). The POSHS has interacting continuous and discrete dynamics with uncertainties. The continuous dynamics are given by a Markov‐jump linear system and the discrete dynamics are defined by a Markov chain whose transition probabilities are dependent on the continuous state via guard conditions. The only information available to the controller are noisy measurements of the continuous state. To solve the optimal control problem, a separable control scheme is applied: the controller estimates the continuous and discrete states of the POSHS using noisy measurements and computes the optimal control input from the state estimates. Since computing both optimal state estimates and optimal control inputs are intractable, this paper proposes computationally efficient algorithms to solve this problem numerically. The proposed hybrid estimation algorithm is able to handle state‐dependent Markov transitions and compute Gaussian‐ mixture distributions as the state estimates. With the computed state estimates, a reinforcement learning algorithm defined on a function space is proposed. This approach is based on Monte Carlo sampling and integration on a function space containing all the probability distributions of the hybrid state estimates. Finally, the proposed algorithm is tested via numerical simulations.

show abstract

Stochastic Hybrid Systems: Application to Communication Networks

Cited by 130 publications

References 40 publications

A Derivative Matching Approach to Moment Closure for the Stochastic Logistic Model

A Derivative Matching Approach to Moment Closure for the Stochastic Logistic Model

Polynomial Stochastic Hybrid Systems

Partially‐observable stochastic hybrid systems (poshss) state estimation and optimal control

Contact Info

Product

Resources

About