Availability and performability analysis for a service degradation process with condition-based preventive maintenance II - estimation and simulation

Abstract

The preventive maintenance is very useful to improve effectively the service availability for software systems with service degradation. In this paper, we present a stochastic model to describe an operational software, which consists of one operating system and multiple applications and provides a service in continuous time. In this paper we examine adaptive estimation problems using a reinforcement learning algorithm which is called the Q-learning. We give the Q-learning algorithms to estimate the optimal software rejuvenation policies that minimise the service availability and maximise the expected service reward per unit time, respectively. Illustrative numerical examples are presented to investigate asymptotic properties of estimators of the optimal software rejuvenation policies.