“…We refer to some applications, such as [1], where the structural condition of storm water pipes is described by five states of a Markov chain, and [2], where the feedback measurements are randomly dropped with a distribution selected from an underlying Markov chain. Moreover, randomness based on Markov chains is also used for decision making (DM) problems, see e.g., [3], which applies a Markov chain to simulate the highway condition index. Specifically, an optimization model of sequential decision making based on the controlled Markov process is developed, and called Markov decision process, cf.…”