Perturbation analysis of inhomogeneous finite Markov chains

Abstract

In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.

RETRACTED: Sensitivity Analysis Based on Markovian Integration by Parts Formula

“…This paper extends some of their results and applies them to calculate the sensitivities of two Markovian models. For other approaches studying the sensitivity and gradient defined for it, we refer to [13] which estimates the gradient for ratio and [14] for inhomogeneous finite Markov chains. However, the sensitivity considered by [14] is only about the parameter of the Markov chain itself, (such as some factor of the transition rate matrix), and it can not be obviously extended for the computation of the commonly-used Greeks.…”

An Improvement of Markovian Integration by Parts Formula and Application to Sensitivity Computation

“…As an important application, our integration by parts formula is used to calculate the sensitivity regarding the transition rate matrix. As we also notice that, this work is achievable by applying the recent research results by Heidergott et al [14], which applies the IPA technique, for the background knowledge of which we refer to [13,15,21] and for the direct applications of IPA, we refer to [11,12]. For the sensitivity computation regarding the changes of transition rate matrix, we show the numerical results by our formula and [14], and it should be rare but inspiring to apply the integration by parts formula and IPA estimator for the same problem of sensitivity analysis, since the former usually deals with the variations of model parameters and the latter handles the perturbation of process parameters.…”

Ergodicity and perturbation bounds forMt/Mt/1 queue with balking, catastrophes, server failures and repairs

“…In [5] DiCrescenzo et al construed the a time-non-homogeneous for double-ended queue subject to catastrophes and repairs, as this is an extension of their previous work in [4]. Some other nonstationary models were studied by a number of authors, see for instance [6,7,8,10,11,23].…”