On the Markov property for nonlinear discrete-time systems with Markovian inputs

Abstract

Abstract-The behavior of a general hybrid system in discrete-time can be represented by a non-linear difference equation, where θ(k) is assumed to be a finite-state Markov chain. An important step in the stability analysis of these systems is to establish the Markov property of (x(k), θ(k)). There are, however, no complete proofs of this property which are simple to understand. This paper aims to correct this problem by presenting a complete and explicit proof, which uses only fundamental measure-theoretical concepts.

On nonlinear discrete-time systems driven by Markov chains

“…However, when hðkÞ is a Markov chain, the Markov property of ðxðkÞ; hðkÞÞ has been either stated without proof [13][14][15], proven through heuristic arguments [16], or proven without sufficient detail [17]. To the best of our knowledge, there is no complete proof of the Markov nature of ðxðkÞ; hðkÞÞ for system (1), when hðkÞ is a Markov chain. This paper aims to address this problem by presenting a complete and explicit proof which uses only fundamental measure-theoretical concepts and follows the probabilistic approach, i.e., it interprets the process ðxðkÞ; hðkÞÞ as a sequence of random vectors and exploits its properties.…”

Mean Square Stability Analysis of Hybrid Jump Linear Systems using a Markov Kernel Approach

“…inputs (HJLS's with Markovian inputs are discussed in [4]). These systems are known to realize Markov chains in a specific metric space [7], [8]. Hence, their long term behavior, i.e., their stability, is governed by the evolution of their densities, which in turn is determined by their associated Markov kernels.…”