Markov Decision Processes with Asymptotic Average Failure Rate Constraint

Boussemart, Michel; Limnios, Nikolaos

doi:10.1081/sta-120037268

Cited by 13 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also present examples of Dynkin's formulae and boundary value problems for controlled additive functionals (CAFs) and controlled geometric Markov renewal chains (CGMRCs). In the literature a CAF is usually called the Markov decision process (see, for example, [3], [7], [8], and [9]). We also mention that the discrete-time case of the stochastic optimal control has been considered in [6].…”

Section: Optimal Control and The Hamilton-jacobi-bellman Equation Formentioning

confidence: 99%

Discrete-Time Semi-Markov Random Evolutions and their Applications

Limnios¹,

Swishchuk²

2013

Adv. Appl. Probab.

View full text Add to dashboard Cite

In this paper we introduce discrete-time semi-Markov random evolutions (DTSMREs) and study asymptotic properties, namely, averaging, diffusion approximation, and diffusion approximation with equilibrium by the martingale weak convergence method. The controlled DTSMREs are introduced and Hamilton-Jacobi-Bellman equations are derived for them. The applications here concern the additive functionals (AFs), geometric Markov renewal chains (GMRCs), and dynamical systems (DSs) in discrete time. The rates of convergence in the limit theorems for DTSMREs and AFs, GMRCs, and DSs are also presented.

show abstract

Section: Optimal Control and The Hamilton-jacobi-bellman Equation Formentioning

confidence: 99%

Discrete-Time Semi-Markov Random Evolutions and their Applications

Limnios¹,

Swishchuk²

2013

Adv. Appl. Probab.

View full text Add to dashboard Cite

show abstract

“…The policy restores the system to a previous, not necessarily AGAN, condition with certain probability. Similarly, Boussemart, Bickard, & Limnios (2001) considered a Markov chain that governs the system degradation, maintenance actions bring the system to a new state with certain probability, the new system state depends on the performed action. More details in this subject can be read in Section 1 .…”

Section: Maintenance In Hmmmentioning

confidence: 99%

Hidden markov models in reliability and maintenance

Gámiz

Limnios

Segovia-García

2023

European Journal of Operational Research

View full text Add to dashboard Cite

“…Both of these reinforcement learning methods assume that every Markov chain induced by a policy is irreducible, which allows only a single recurrent class as with ergodic and unichain assumptions described earlier. The Lagrangian approach has also been applied to specific stochastic policy linear programming formulations relevant to aircraft maintenance problems where the asymptotic failure is to be kept below some small threshold (Boussemart & Limnios, 2004;Boussemart et al, 2002).…”

Section: Related Workmentioning

confidence: 99%

“…Steady-state planning has applications in several areas, such as deriving maintenance plans for various systems, including aircraft maintenance, where the asymptotic failure rate of components must be kept below some small threshold (Boussemart & Limnios, 2004;Boussemart, Limnios, & Fillion, 2002). Optimal routing problems for communication networks have also been proposed in which data throughput must be maximized subject to constraints on average delay and packet drop metrics (Lazar, 1983).…”

Section: Introductionmentioning

confidence: 99%

Steady-State Planning in Expected Reward Multichain MDPs

Atia¹,

Beckus²,

Alkhouri³

et al. 2021

jair

View full text Add to dashboard Cite

The planning domain has experienced increased interest in the formal synthesis of decision-making policies. This formal synthesis typically entails finding a policy which satisfies formal specifications in the form of some well-defined logic. While many such logics have been proposed with varying degrees of expressiveness and complexity in their capacity to capture desirable agent behavior, their value is limited when deriving decision-making policies which satisfy certain types of asymptotic behavior in general system models. In particular, we are interested in specifying constraints on the steady-state behavior of an agent, which captures the proportion of time an agent spends in each state as it interacts for an indefinite period of time with its environment. This is sometimes called the average or expected behavior of the agent and the associated planning problem is faced with significant challenges unless strong restrictions are imposed on the underlying model in terms of the connectivity of its graph structure. In this paper, we explore this steady-state planning problem that consists of deriving a decision-making policy for an agent such that constraints on its steady-state behavior are satisfied. A linear programming solution for the general case of multichain Markov Decision Processes (MDPs) is proposed and we prove that optimal solutions to the proposed programs yield stationary policies with rigorous guarantees of behavior.

show abstract

Markov Decision Processes with Asymptotic Average Failure Rate Constraint

Cited by 13 publications

References 7 publications

Discrete-Time Semi-Markov Random Evolutions and their Applications

Discrete-Time Semi-Markov Random Evolutions and their Applications

Hidden markov models in reliability and maintenance

Steady-State Planning in Expected Reward Multichain MDPs

Contact Info

Product

Resources

About