Estimating the stationary distribution of a Markov chain

Athreya, Krishna B.; Majumdar, Mukul

doi:10.1007/978-3-662-05858-9_33

Cited by 6 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Contrarily, if ι = 1 in (27) for a large region (likely to be visited asymptotically) in the state-space, then g a is small. Numerical methods to estimate χ inv can be found, e.g., in [50], [51]. In the example of Section VI we obtain a good approximation of g a by running Monte-Carlo simulations.…”

Section: Quantifying the Performance Improvementsmentioning

confidence: 99%

Rollout Event-Triggered Control: Beyond Periodic Control Performance

Antunes

Heemels

2014

IEEE Trans. Automat. Contr.

139

View full text Add to dashboard Cite

Cyber-Physical Systems (CPSs) resulting from the interconnection of computational, communication, and control (cyber) devices with physical processes are wide spreading in our society. In several CPS applications it is crucial to minimize the communication burden, while still providing desirable closed-loop control properties. To this effect, a promising approach is to embrace the recently proposed event-triggered control paradigm, in which the transmission times are chosen based on well-defined events, using state information. However, few general eventtriggered control methods guarantee closed-loop improvements over traditional periodic transmission strategies. Here, we provide a new class of event-triggered controllers for linear systems which guarantee better quadratic performance than traditional periodic time-triggered control using the same average transmission rate. In particular, our main results explicitly quantify the obtained performance improvements for quadratic average cost problems. The proposed controllers are inspired by rollout ideas in the context of dynamic programming. Index Terms-Approximate dynamic programming, control over communications, event-triggered control, Markov processes, stochastic optimal control. I. INTRODUCTIONC YBER devices capable of sensing, processing, and communicating information of interest are wide spreading in our society, creating new opportunities to make our physical processes operate exceedingly better. In fact, the number of applications in which communication, computation and control elements (the cyber part) go hand in hand with motion, energy, climate, and human processes (the physical part) is steadily growing in intelligent transportation, smart buildings, energy networks, healthcare, and robotics (see, e.g., [2]-[6], respectively). To meet the challenges arising in many of these applications the traditional separation-of-concerns principle in designing control, communication, and computational algorithms must be abandoned in favor of an integrated approach. This can lead to dramatic communication and computation savings in control applications, which is crucial to prevent Manuscript

show abstract

Section: Quantifying the Performance Improvementsmentioning

confidence: 99%

Rollout Event-Triggered Control: Beyond Periodic Control Performance

Antunes

Heemels

2014

IEEE Trans. Automat. Contr.

139

View full text Add to dashboard Cite

show abstract

“…For a unified exposition of √ n-consistent estimation and other results see Athreya and Majumdar (2002).…”

Section: An Estimation Problemmentioning

confidence: 99%

Random dynamical systems: a review

Bhattacharya

Majumdar

2003

Economic Theory

Self Cite

View full text Add to dashboard Cite

This paper provides a review of some results on the stability of random dynamical systems and indicates a number of applications to stochastic growth models, linear and non-linear time series models, statistical estimation of invariant distributions, and random iterations of quadratic maps. Copyright Springer-Verlag Berlin/Heidelberg 2003Random dynamical systems, Stability, Splitting, Contractions, Stochastic growth, Time series, Estimation.,

show abstract

Stationary measures for some Markov chain models in ecology and economics

Athreya

2003

Economic Theory

Self Cite

View full text Add to dashboard Cite

Let $F \equiv \{f : f : [0, \infty) \rightarrow [0, \infty), f (0)=0, f$ continuous, $\lim\limits_{x \downarrow 0} \frac{f(x)}{x}=C$ exists in $(0, \infty), 0 < g (x) \equiv \frac{f(x)}{C x} < 1$ for x in $(0, \infty)\}$ . Let $\{f_j\}_{j \geq 1}$ be an i.i.d. sequence from F and X 0 be a nonnegative random variable independent of $\{f_j\}_{j \geq 1}$ . Let $\{X_n\}_{n \geq 0}$ be the Markov chain generated by the iteration of random maps $\{f_j\}_{j \geq 1}$ by $X_{n + 1}=f_{n + 1} (X_n), n \geq 0$ . Such Markov chains arise in population ecology and growth models in economics. This paper studies the existence of nondegenerate stationary measures for {X n }. A set of necessary conditions and two sets of sufficient conditions are provided. There are some convergence results also. The present paper is a generalization of the work on random logistics maps by Athreya and Dai (2000). Copyright Springer-Verlag Berlin/Heidelberg 2003Population models, Random maps, Markov chains, Stationary measures.,

show abstract

Estimating the stationary distribution of a Markov chain

Cited by 6 publications

References 17 publications

Rollout Event-Triggered Control: Beyond Periodic Control Performance

Rollout Event-Triggered Control: Beyond Periodic Control Performance

Random dynamical systems: a review

Stationary measures for some Markov chain models in ecology and economics

Contact Info

Product

Resources

About