Imprecise continuous-time Markov chains

Abstract

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models computationally tractable, they rely on a number of assumptions that-as is well known-may not be realistic for the domain of application; in particular, the ability to provide exact numerical parameter assessments, and the applicability of time-homogeneity and the eponymous Markov property. In this work, we extend these models to imprecise continuous-time Markov chains (ICTMC's), which are a robust generalisation that relaxes these assumptions while remaining computationally tractable. More technically, an ICTMC is a set of "precise" continuous-time finite-state stochastic processes, and rather than computing expected values of functions, we seek to compute lower expectations, which are tight lower bounds on the expectations that correspond to such a set of "precise" models. Note that, in contrast to e.g. Bayesian methods, all the elements of such a set are treated on equal grounds; we do not consider a distribution over this set. Together with the conjugate notion of upper expectation, the bounds that we provide can then be intuitively interpreted as providing best- and worst-case scenarios with respect to all the models in our set of stochastic processes. The first part of this paper develops a formalism for describing continuous-time finite-state stochastic processes that does not require the aforementioned simplifying assumptions. Next, this formalism is used to characterise ICTMC's and to investigate their properties. The concept of lower expectation is then given an alternative operator-theoretic characterisation, by means of a lower transition operator, and the properties of this operator are investigated as well. Finally, we use this lower transition operator to derive tractable algorithms (with polynomial runtime complexity w.r.t. the maximum numerical error) for computing the lower expectation of functions that depend on the state at any finite number of time points. (C) 2017 Elsevier Inc. All rights reserved

Computing Inferences for Large-Scale Continuous-Time Markov Chains by Combining Lumping with Imprecision

“…We assume that the stochastic process that models our beliefs about the system, denoted by (X t ) t∈IR ≥0 , is a continuous-time Markov chain (CTMC) that is homogeneous. For a thorough treatment of the terminology and notation concerning CTMCs, we refer to [1,11,13]. Due to length constraints, we here limit ourselves to the bare necessities.…”

“…Note how strikingly (8) resembles (4). Analogous to the precise case, one needs numerical methods-see for instance [6] or [11,Sect. 8.2]-to approximateT t g because analytically evaluating the limit in (8) is, at least in general, impossible.…”

“…We often rely on results from Krak et al [11] when proving our results. In order to facilitate the use of these results, we try to adhere to the notation used in [11] as much as possible.…”

“…[11] focus on three nested sets of processes, all characterised by a nonempty set of initial distributions M and a non-empty bounded set of transition rate matrices Q ⊆ R(X ). More specifically, they collect in P W Q,M all stochastic processes that are: (i) well-behaved, a technical condition [11,Definition 4.4]; (ii) consistent with Q, in the sense that at all times the "instantaneous transition rate matrix" is contained in Q [11, Definition 6.1]; and (iii) consistent with M, in the sense that M contains the initial distribution [11,Definition 6.2]. Similarly, P WM Q,M (P WHM Q,M ) contains all well-behaved (homogeneous) Markov processes that are consistent with Q and M. These sets are clearly nested, in the sense that…”

“…In this contribution, we address this problem using imprecise continuoustime Markov chains [5,11,16]. In particular, we outline an approach to determine guaranteed lower and upper bounds on marginal and limit expectations using the reduced state space.…”