Computing Continuous-Time Markov Chains as Transformers of Unbounded Observables

Abstract

This paper provides broad sufficient conditions for the computability of time-dependent averages of stochastic processes of the form f (Xt) where Xt is a continuous-time Markov chain (CTMC), and f is a real-valued function (aka an observable). We consider chains with values in a countable state space S, and possibly unbounded f s. Observables are seen as generalised predicates on S and chains are interpreted as transformers of such generalised predicates, mapping each observable f to a new observable Ptf defined as (Ptf)(x) = Ex(f (Xt)), which represents the mean value of f at time time t as a function of the initial state x. We obtain three results. First, the well-definedness of this operator interpretation is obtained for a large class of chains and observables by restricting Pt to judiciously chosen rescalings of the basic Banach space C0(S) of S-indexed sequences which vanish at infinity. We prove, under appropriate assumptions, that the restricted family Pt forms a strongly continuous operator semigroup (equivalently the time evolution map t → Pt is continuous w.r.t. the usual topology on bounded operators). The computability of the time evolution map follows by generic arguments of constructive analysis. A key point here is that the assumptions are flexible enough to accommodate unbounded observables, and we give explicit examples of such using stochastic Petri nets and stochastic string rewriting. Thirdly, we show that if the rate matrix (aka the q-matrix) of the CTMC is locally algebraic on a subspace containing f , the time evolution of projections t → (Ptf)(x) is PTIME computable for each x. These results provide a functional analytic alternative to Monte Carlo simulation as test bed for mean-field approximations, moment closure, and similar techniques that are fast, but lack absolute error guarantees.

Rate Equations for Graphs

“…Some progress on going beyond the formal point of view and obtaining existence theorems for solutions of REs for graphs were reported in Ref. [14]. Another limitation is the accuracy of MFAs once truncated (as they must be if one wants to plug them in an ODE solver).…”

“…giving us an equation for the rate of change of the mean of f (X(t)). Following this general recipe gives us the GRE for graphs immediately from (14).…”

“…In the present paper, we elide the subtle issues of ensuring that the system of interest actually satisfies this equation. See the work of Spieksma[41] for the underlying mathematics or our previous work[14], which additionally considers computability of the solutions to arbitrary precision.…”

Rate Equations for Graphs

“…Some progress on going beyond the formal point of view and obtaining existence theorems for solutions of REs for graphs were reported in Ref. [8]. Another limitation is the accuracy of MFAs once truncated (as they must be if one wants to plug them in an ODE solver).…”

“…The crucial ingredient that makes backward modularity possible is that both matches and rules are monos. Because rules are (partial) monos, we can reverse α and β in (8), and the resulting diagram still commutes (though it is no longer a derivation in general). The existence and uniqueness of f 2 is then a direct consequence of the universal property of ( 9), seen as a PO.…”