Microcanonical conditioning of Markov processes on time-additive observables

Abstract

The recent study by B. De Bruyne, S. N. Majumdar, H. Orland and G. Schehr [arXiv:2110.07573], concerning the conditioning of the Brownian motion and of random walks on global dynamical constraints over a finite time-window T , is reformulated as a general framework for the 'microcanonical conditioning' of Markov processes on time-additive observables. This formalism is applied to various types of Markov processes, namely discrete-time Markov chains, continuous-time Markov jump processes and diffusion processes in arbitrary dimension. In each setting, the time-additive observable is also fully general, i.e. it can involve both the time spent in each configuration and the elementary increments of the Markov process. The various cases are illustrated via simple explicit examples. Finally, we describe the link with the 'canonical conditioning' based on the generating function of the time-additive observable for finite time T , while the regime of large time T allows to recover the standard large deviation analysis of time-additive observables via the deformed Markov operator approach.

Generating stochastic trajectories with global dynamical constraints

“…It would be interesting to see what this result may imply for algorithms based on reinforcement learning where this question was recently raised [20]. Finally, we note that our approach has recently been used by Monthus [80] to provide a general framework for the so-called 'micro-canonical conditioning' of Markov processes on time-additive observables that led to further applications.…”

“…where P c (x c , t|T f , t f ) and P c (A c , t|T f , t f ) are the marginal distributions given in (80). By substituting the joint distribution from ( 79), the double integral ( 81) can be easily evaluated numerically and compared to numerical simulations using the effective Langevin equation ( 77), finding very good agreement.…”

Optimal Resetting Brownian Bridges

“…Our first goal is to construct a rejection-free algorithm to generate an RBB with the correct statistical weight. Generating constrained stochastic Markov processes was initially studied in the probability literature [73,74] and more recently it has emerged as a vibrant research area by itself in the context of sampling rare/constrained trajectories with applications ranging from chemistry and biology all the way to particle physics [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94]. In the context of the RBB, a naive solution would be to generate free RBM paths and discard the ones that do not satisfy the bridge constraint x B (t f ) = 0.…”