Relative entropy and exponential deviation bounds for general Markov chains

Kontoyiannis, Ioannis; Lastras-Montaño, L.A.; Meyn, Sean

doi:10.1109/isit.2005.1523607

Cited by 33 publications

(35 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proof of this result only requires slight modifications from the i.i.d. case, requiring a more general concentration bound of the type to the stationary distribution [31] and a Berry-Esseen theorem for weakly-dependent processes such as Markov processes [32].…”

Section: B Specializations To Various State Modelsmentioning

confidence: 99%

Second-Order Coding Rates for Channels With State

Tomamichel

Tan

2014

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We study the performance limits of state-dependent discrete memoryless channels with a discrete state available at both the encoder and the decoder. We establish the ε-capacity as well as necessary and sufficient conditions for the strong converse property for such channels when the sequence of channel states is not necessarily stationary, memoryless or ergodic. We then seek a finer characterization of these capacities in terms of second-order coding rates. The general results are supplemented by several examples including i.i.d. and Markov states and mixed channels.

show abstract

Section: B Specializations To Various State Modelsmentioning

confidence: 99%

Second-Order Coding Rates for Channels With State

Tomamichel

Tan

2014

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Niemiro and Pokarowski in [39] give results for relative precision estimation. For uniformly ergodic chains and bounded function f , Hoeffding type inequalities are available in [21,32,33] and can easily lead to (2). Tail inequalities for bounded functionals of Markov chains that are not uniformly ergodic were considered in [11,1,13] using regeneration techniques.…”

Section: Introductionmentioning

confidence: 99%

Rigorous confidence bounds for MCMC under a geometric drift condition

Łatuszyński

Niemiro

2011

Journal of Complexity

View full text Add to dashboard Cite

We assume a drift condition towards a small set and bound the mean square error of estimators obtained by taking averages along a single trajectory of a Markov chain Monte Carlo algorithm. We use these bounds to construct fixed-width nonasymptotic confidence intervals. For a possibly unbounded function $f:\stany \to R,$ let $I=\int_{\stany} f(x) \pi(x) dx$ be the value of interest and $\hat{I}_{t,n}=(1/n)\sum_{i=t}^{t+n-1}f(X_i)$ its MCMC estimate. Precisely, we derive lower bounds for the length of the trajectory $n$ and burn-in time $t$ which ensure that $$P(|\hat{I}_{t,n}-I|\leq \varepsilon)\geq 1-\alpha.$$ The bounds depend only and explicitly on drift parameters, on the $V-$norm of $f,$ where $V$ is the drift function and on precision and confidence parameters $\varepsilon, \alpha.$ Next we analyse an MCMC estimator based on the median of multiple shorter runs that allows for sharper bounds for the required total simulation cost. In particular the methodology can be applied for computing Bayesian estimators in practically relevant models. We illustrate our bounds numerically in a simple example

show abstract

“…where δ 0 is the gap of the contraction of the Markov chain and C is an absolute explicit constant. Such explicit inequalities where obtained by Glynn and Ormoneit [GO02] and Kontoyiannis, Lastras-Montaño and Meyn [KLMM05] using the characterization of uniform ergodicity by the Doeblin minorization condition; they obtain a non-optimal quadratic dependency on the gap (although their results are stated with another, directly related parameter β). More recently, an effective concentration inequality with the optimal dependency on δ 0 and better constants than ours was obtained by Paulin [Pau15] (Corollary 2.10).…”

Section: Chains With Doeblin's Minorizationmentioning

confidence: 92%

Effective Berry–Esseen and concentration bounds for Markov chains with a spectral gap

Kloeckner¹

2019

Ann. Appl. Probab.

View full text Add to dashboard Cite

Applying quantitative perturbation theory for linear operators, we prove non-asymptotic bounds for Markov chains whose transition kernel has a spectral gap in an arbitrary Banach algebra of functions X . The main results are concentration inequalities and Berry-Esseen bounds, obtained assuming neither reversibility nor "warm start" hypothesis: the law of the first term of the chain can be arbitrary. The spectral gap hypothesis is basically a uniform X -ergodicity hypothesis, and when X consist in regular functions this is weaker than uniform ergodicity. We show on a few examples how the flexibility in the choice of function space can be used. The constants are completely explicit and reasonable enough to make the results usable in practice, notably in MCMC methods. 1 Here and in the sequel, we write indifferently µ(f ) or f dµ for the integral of f with respect to the measure µ.bounds, both for the Law of Large Numbers (1) ("concentration inequalities") and for the CLT ("Berry-Esseen bounds").A word on effectivity In this paper, the emphasis will be on effective bounds, i.e. given an explicit sample size n, one should be able to deduce from the bound that the quantity being considered lies in some explicit interval around its limit with at least some explicit probability. In other words, the result should be non-aymptotic and all constants should be made explicit. The motivations for this are at least twofold.First, in practical applications of the Markov chain Monte-Carlo (MCMC) method, where one uses (1) to estimate the integral µ 0 (ϕ), effective results are needed to obtain proven convergence of a given precision. MCMC methods are important when the measure of interest is either unknown, or difficult to sample independently (e.g. uniform in a convex set in large dimension), but happens to be the stationary measure for an easily simulated Markov chain. The Metropolis-Hastings algorithm for example makes it possible to deal with an absolutely continuous measure whose density is only known up to the normalization constant.A second, more theoretical motivation is that the constants appearing in limit theorem depend on a number of parameters (e.g. the mixing speed of the Markov chain, the law of X 0 , etc.). When the constants are not made explicit, one may not be able to deduce from the result how the convergence speed changes when some parameter approaches the limit of the domain where the result is valid (e.g. when the spectral gap tends to 0).There are many works proving concentration inequalities and (to a lesser extent) Berry-Esseen bounds for Markov chains, under a variety of assumptions, and we will only mention a small number of them. To explain the purpose of this article, let us discuss briefly three directions. Previous works (1): total variation convergenceThe first direction is mainly motivated by MCMC; we refer to [RR + 04] for a detailed introduction to the topic.The Markov chains being considered are usually ergodic (either uniformly, which corresponds to a spectral gap on L ∞ , or geometrically);...

show abstract

Relative entropy and exponential deviation bounds for general Markov chains

Cited by 33 publications

References 14 publications

Second-Order Coding Rates for Channels With State

Second-Order Coding Rates for Channels With State

Rigorous confidence bounds for MCMC under a geometric drift condition

Effective Berry–Esseen and concentration bounds for Markov chains with a spectral gap

Contact Info

Product

Resources

About