In this article we give an overview of Markov models that are used for describing dynamics of operating environments in reliability analysis. We present both continuous‐ and discrete‐time Markov chains as well as diffusion processes. We also consider Markov modulated stochastic processes that are known as
hidden Markov models
and discuss their properties. More specifically, we give an overview of Markov modulated Poisson and Markov modulated Bernoulli processes.