Regenerative Markov Chain Monte Carlo for Any Distribution

Minh, Do Le; Minh, David D. L.; Nguyen, Andrew

doi:10.1080/03610918.2011.615433

Cited by 10 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea of identifying regeneration times within a given MCMC sampler goes back to Mykland et al [1995], using the very elegant splitting technique of Nummelin [1978]. The area has continued to develop actively, as seen for instance in the contributions of Gilks et al [1998], Hobert et al [2002], Brockwell and Kadane [2005], Minh et al [2012], Lee et al [2014]. The idea of hybridising separate dynamics has also had a long history, see, for instance, Tierney [1996], Murdoch and Green [1998], Murdoch [2000], although these typically involve combining separate MCMC chains which are already themselves π-invariant.…”

Section: Introductionmentioning

confidence: 99%

Regeneration-enriched Markov processes with application to Monte Carlo

Wang,

Pollock,

Roberts

et al. 2019

Preprint

View full text Add to dashboard Cite

We study a class of Markov processes comprising local dynamics governed by a fixed Markov process which are enriched with regenerations from a fixed distribution at a state-dependent rate. We give conditions under which such processes possess a given target distribution as their invariant measures, thus making them amenable for use within Monte Carlo methodologies. Enrichment imparts a number of desirable theoretical and methodological properties: Since the regeneration mechanism can compensate the choice of local dynamics, while retaining the same invariant distribution, great flexibility can be achieved in selecting local dynamics, and mathematical analysis is simplified. In addition we give straightforward conditions for the process to be uniformly ergodic and possess a coupling from the past construction that enables exact sampling from the invariant distribution. More broadly, the sampler can also be used as a recipe for introducing rejection-free moves into existing Markov Chain Monte Carlo samplers in continuous time.

show abstract

Section: Introductionmentioning

confidence: 99%

Regeneration-enriched Markov processes with application to Monte Carlo

Wang,

Pollock,

Roberts

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The (non-parametric) ACTMCs form a subclass of Markov regenerative processes (MRP) [2,8,23]. Alternatively, ACTMCs can be also understood as a generalized semi-Markov processes (GSMPs) with at most one non-exponential event enabled in each state or as bounded stochastic Petri nets (SPNs) [12] with at most one non-exponential transition enabled in any reachable marking [8].…”

Section: Introductionmentioning

confidence: 99%

Mean-payoff Optimization in Continuous-time Markov Chains with Parametric Alarms

Baier

Dubslaff

Korenčiak

et al. 2019

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

Continuous-time Markov chains with alarms (ACTMCs) allow for alarm events that can be non-exponentially distributed. Within parametric ACTMCs, the parameters of alarm-event distributions are not given explicitly and can be subject of parameter synthesis. An algorithm solving the ε-optimal parameter synthesis problem for parametric ACTMCs with long-run average optimization objectives is presented. Our approach is based on reduction of the problem to finding long-run average optimal strategies in semi-Markov decision processes (semi-MDPs) and sufficient discretization of parameter (i.e., action) space. Since the set of actions in the discretized semi-MDP can be very large, a straightforward approach based on explicit action-space construction fails to solve even simple instances of the problem. The presented algorithm uses an enhanced policy iteration on symbolic representations of the action space. The soundness of the algorithm is established for parametric ACTMCs with alarm-event distributions satisfying four mild assumptions that are shown to hold for uniform, Dirac and Weibull distributions in particular, but are satisfied for many other distributions as well. An experimental implementation shows that the symbolic technique substantially improves the efficiency of the synthesis algorithm and allows to solve instances of realistic size.

show abstract

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms

Baier

Dubslaff

Korenčiak

et al. 2017

Quantitative Evaluation of Systems

View full text Add to dashboard Cite

Abstract. Continuous-time Markov chains with alarms (ACTMCs) allow for alarm events that can be non-exponentially distributed. Within parametric ACTMCs, the parameters of alarm-event distributions are not given explicitly and can be subject of parameter synthesis. An algorithm solving the ε-optimal parameter synthesis problem for parametric ACTMCs with long-run average optimization objectives is presented. Our approach is based on reduction of the problem to finding long-run average optimal strategies in semi-Markov decision processes (semi-MDPs) and sufficient discretization of parameter (i.e., action) space. Since the set of actions in the discretized semi-MDP can be very large, a straightforward approach based on explicit action-space construction fails to solve even simple instances of the problem. The presented algorithm uses an enhanced policy iteration on symbolic representations of the action space. The soundness of the algorithm is established for parametric ACTMCs with alarm-event distributions satisfying four mild assumptions that are shown to hold for uniform, Dirac and Weibull distributions in particular, but are satisfied for many other distributions as well. An experimental implementation shows that the symbolic technique substantially improves the efficiency of the synthesis algorithm and allows to solve instances of realistic size.

show abstract

Regenerative Markov Chain Monte Carlo for Any Distribution

Cited by 10 publications

References 21 publications

Regeneration-enriched Markov processes with application to Monte Carlo

Regeneration-enriched Markov processes with application to Monte Carlo

Mean-payoff Optimization in Continuous-time Markov Chains with Parametric Alarms

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms

Contact Info

Product

Resources

About