Remarks on Non-Markov Processes

Abstract

It is emphasized that non-Markovian processes, which occur for instance in the case of colored noise, cannot be considered merely as corrections to the class of Markov processes but require special treatment. Many familiar concepts, such as rst-passage times, no longer apply and need be reconsidered. Several methods of dealing with non-Markov processes are discussed. As an example a recent application to the transport of ions through a membrane is brie y mentioned.

Analysis of the spatial and temporal distributions between successive earthquakes: Nonextensive statistical mechanics viewpoint

“…All other quantities which are concerned with thermodynamics may be derived from it and also inherit the nonextensivity (nonintensivity) property. Nonextensive statistical mechanics seems to be a suitable candidate for describing systems with finite degrees of freedom [18,19], complex systems, self organized criticality, long range classical Hamiltonian systems and nonMarkovian processes with long range memory [11,20]. The physical distributions are obtained by maximization of the entropy under appropriate constraints,…”

Reinforcement Learning in Non-Markov Conservative Environment Using an Inductive Qualitative Model

“…The term "non-Markov process" represents all random processes with the exception of the very small minority that happens to have the Markov property. 3 It is true that this minority has been extensively studied, especially in the reinforcement-learning field, but it is not proper to treat non-Markov processes as modifications or corrections of the Markov process. What is more characteristic is that this 'small minority' exhibits unusable features such that even the simplest Markov periodic process with two mutually exchange of states with a probability of one does not have a stable limit distribution, 1 that is it cannot be used for system-long term-prediction calculation.…”

Correlation effects in the stochastic Landau-Lifshitz-Gilbert equation

“…Based on Eq. (11) the appropriate joint probability distribution is defined by [24,25], for a more general discussion compare also [26]:…”