Tutorial on Structured Continuous-Time Markov Processes

Abstract

A continuous-time Markov process (CTMP) is a collection of variables indexed by a continuous quantity, time. It obeys the Markov property that the distribution over a future variable is independent of past variables given the state at the present time. We introduce continuous-time Markov process representations and algorithms for filtering, smoothing, expected sufficient statistics calculations, and model estimation, assuming no prior knowledge of continuous-time processes but some basic knowledge of probability and statistics. We begin by describing "flat" or unstructured Markov processes and then move to structured Markov processes (those arising from state spaces consisting of assignments to variables) including Kronecker, decision-diagram, and continuous-time Bayesian network representations. We provide the first connection between decision-diagrams and continuoustime Bayesian networks. Tutorial GoalsThis tutorial is intended for readers interested in learning about continuous-time Markov processes, and in particular compact or structured representations of them. It is assumed that the reader is familiar with general probability and statistics and has some knowledge of discrete-time Markov chains and perhaps hidden Markov model algorithms.While this tutorial deals only with Markovian systems, we do not require that all variables be observed. Thus, hidden variables can be used to model long-range interactions among observations. In these models, at any given instant the assignment to all state variables is sufficient to describe the future evolution of the system. The variables themselves real-valued (continuous) times. We consider evidence or observations that can be regularly spaced, irregularly spaced, or continuous over intervals. These evidence patterns can change by model variable and time.We deal exclusively with discrete-state continuous-time systems. Real-valued variables are important in many situations, but to keep the scope manageable, we will not treat them here. We refer to the work of Särkkä (2006) for a machine-learning-oriented treatment of filtering and smoothing in such models. The literature on parameter estimation is more scattered. We will further constrain our discussion to systems with finite states, although many of the concepts can be extended to countably infinite state systems.We will be concerned with two main problems: inference and learning (parameter estimation). These were chosen as those most familiar to and applicable for researchers in artificial intelligence. At points we will also discuss the computation of steady-state properties, especially for model for which most research concentrates on this computation.

Integration of Anatomy Ontologies and Evo-Devo Using Structured Markov Models Suggests a New Framework for Modeling Discrete Phenotypic Traits

“…This class of models (Nodelman et al 2002), also known as continuous-time Bayesian Networks (Shelton and Ciardo 2014), arises from traditional MMs. The only difference between the two is that SMMs are equipped with a specific parameterization of the rate matrix to model dependencies.…”

Integration of anatomy ontologies and evo-devo using structured Markov models suggests a new framework for modeling discrete phenotypic traits

“…Structured Markov Models (SMM). -This class of models (Nodelman et al 2002), also known as continuous-time Bayesian Networks (Shelton and Ciardo 2014), arises from traditional Markov models. The only difference between the two is that SMMs are equipped with a specific parameterization of the rate matrix to model dependencies.…”

“…Amalgamation of characters. -The properties of SMM (Shelton and Ciardo 2014) allow mathematically valid amalgamation (Box 1, section A) of any number of characters thus removing distinction between character and character state. The amalgamation is especially straightforward when characters are independent (the dependent cases are reviewed in the next section).…”

Robust statistical processing of TDOA estimates for distant speaker diarization

“…Assuming an approximate average speaker duration of 2.5 s [4] and the TDOA frame increment of 62.5 ms, then a 21 = 0.025. This ratio is derived from the fact that the number of steps in the same state is geometrically distributed [24] and its expected value is 1/(1 − a qq ) for q ∈ {1, 2, · · · , N spk }. Therefore 1/(1−a qq ) is set to be the average speaker duration in frames.…”