Master equations and the theory of stochastic path integrals

Abstract

Abstract. This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. Since the 1930s, master equations have served as a fundamental tool to understand the role of fluctuations in complex biological, chemical, and physical systems. Despite their simple appearance, analyses of masters equations most often rely on lownoise approximations such as the Kramers-Moyal or the system size expansion, or require ad-hoc closure schemes for the derivation of low-order moment equations. We focus on numerical and analytical methods going beyond the low-noise limit and provide a unified framework for the study of master equations. After deriving the forward and backward master equations from the Chapman-Kolmogorov equation, we show how the two master equations can be cast into either of four linear partial differential equations (PDEs). Three of these PDEs are discussed in detail. The first PDE governs the time evolution of a generalized probability generating function whose basis depends on the stochastic process under consideration. Spectral methods, WKB approximations, and a variational approach have been proposed for the analysis of the PDE. The second PDE is novel and is obeyed by a distribution that is marginalized over an initial state. It proves useful for the computation of mean extinction times. The third PDE describes the time evolution of a "generating functional", which generalizes the so-called Poisson representation. Subsequently, the solutions of the PDEs are expressed in terms of two path integrals: a "forward" and a "backward" path integral. Combined with inverse transformations, one obtains two distinct path integral representations of the conditional probability distribution solving the master equations. We exemplify both path integrals in analysing elementary chemical reactions. Moreover, we show how a well-known path integral representation of averaged observables can be recovered from them. Upon expanding the forward and the backward path integrals around stationary paths, we then discuss and extend a recent method for the computation of rare event probabilities. Besides, we also derive path integral representations for processes with continuous state spaces whose forward and backward master equations admit Kramers-Moyal expansions. A truncation of the backward expansion at the level of a diffusion approximation recovers a classic path integral representation of the (backward) Fokker-Planck equation. One can rewrite this path integral in terms of an Onsager-Machlup function and, for purely diffusive Brownian motion, it simplifies to the path integral of Wiener. To make this review accessible to a broad community, we have used the language of probability theory rather than quantum (field) theory and do not assume any knowledge of the latter. The probabilistic structures underpinning various technical concepts, such as coherent states, the Doi-shift, and normal-ordered observables, are thereby made explicit. arXiv:1609.02849v2 [cond-mat...

Stochastic thermodynamics of all-to-all interacting many-body systems

“…Using Eqs. (37)-(40), (66) and (67), the fluctuating entropy production (41) along the forward process can be rewritten as follows…”

“…In this section, the question of how to infer the fluctuations in the macroscopic limit, N → ∞, will be addressed. To shed light on this question, we will employ the Martin-Siggia-Rose formalism [65,66] which equivalently represents the Markovian jump process via a path integral. As will be demonstrated in the following, this path-integral formalism allows to establish a fluctuating description valid at macroscopic scales in the large deviation sense [67], that is for fluctuations that scale exponentially with the number of units N .…”

Unfolding tagged particle histories in single-file diffusion: exact single- and two-tag local times beyond large deviation theory

“…with the corresponding stochastic action functional t x [ ( )]of the continuous [22,30] or discrete state-space [31] Markov process t x( ), and where we introduced the Dirac delta function x d ( ). By means of a straightforward vectorial generalization of the trotterization of the the path integral (A2) in [19,22] (for the backward and forward approach, respectively), one finds that the generating function, corresponding to the Laplace transform…”

Computation of stationary distributions in stochastic models of cellular processes with molecular memory

“…For a biochemical reaction system, if the stochastic motion of the reactants is assumed to be uninfluenced by previous states and only by the current state [3,4], then the stochastic effect can be well revealed by analyzing the chemical master equation (CME) [3][4][5]. This Markovian assumption has also led to important successes in the description of many stochastic processes on networks [3][4][5][6][7][8].…”