Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation

Παπακωνσταντίνου, Κωνσταντίνος; Shinozuka, Masanobu

doi:10.1016/j.ress.2014.04.006

Cited by 106 publications

(47 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By following policy π*, the agent optimally explores and exploits in the POMDP, without the need of explicitly computing any VoI. Component-level inspection scheduling can be incorporated in the POMDP framework by including exploratory actions [2][3][4][5][6][7]. In the example reported in [6], action "visual inspection" is listed in the available options, together with action "repair".…”

Section: Information Gathering In Pomdpsmentioning

confidence: 99%

“…POMDPs include probabilistic models of degradation, cost and observation depending on the management decisions, and are rooted in Bayesian analysis. Both Markov Decision Processes (MDPs) and POMDPs have been extensively investigated for IM applications [2][3][4][5][6][7][8][9][10][11][12][13], due Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/ress to the computational efficiency of dynamic programming. A seminal introduction on VoI analysis is provided by the book of Raiffa and Schlaifer [14], while applications to management of structural and infrastructure systems is provided by Pozzi and Der Kiureghian [15], Straub [16], Zonta et al [17], and Malings and Pozzi [18]; most applications refer to a single-decision making problem, however assessment of the VoI in sequential decision making is also presented in [16,19,20].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling

Memarzadeh

Pozzi

2016

Reliability Engineering & System Safety

View full text Add to dashboard Cite

Section: Information Gathering In Pomdpsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling

Memarzadeh

Pozzi

2016

Reliability Engineering & System Safety

View full text Add to dashboard Cite

“…Long-term O&M under uncertainty can be modeled as a POMDP. 16,37 In this framework, the condition of a component evolves stochastically in time, following a Markov process, depending on the maintenance actions, and costs depend on the current condition and maintenance action. The condition state is not necessarily completely observable.…”

Section: Background and Notation For Pomdp Frameworkmentioning

confidence: 99%

What makes long‐term monitoring convenient? A parametric analysis of value of information in infrastructure maintenance

Pozzi

2019

Struct Control Health Monit

View full text Add to dashboard Cite

Summary Information collected by monitoring systems can provide a significant economic benefit to the operation and maintenance of infrastructure components only under specific conditions. The information has to be precise, not redundant, related to relevant decision problems under uncertainty as, for example, the appropriate scheduling of maintenance actions, and the decision maker needs to be able to process that information and react timely. All these considerations can be naturally embedded in the value of information (VoI), a utility‐based metric for assessing the impact of the additional information in decision making under uncertainty. In this paper, we investigate the relation between the VoI and key features of the monitoring system, of the component deterioration and of the decision‐making process, including measure accuracy and availability, deterioration rates, damage predictability, reaction time, maintenance costs, and the economic discount factor. By leveraging previous work, we model the maintenance process as a partially observable Markov decision process, and we compute the VoI of long‐term monitoring. Our proposed framework allows for a detailed quantitative analysis on the joint effects of these features and can be useful to identify conditions when the benefit of monitoring is high, to assign priorities among components that deserve to be instrumented or to optimize the allocation of resources to monitoring efforts.

show abstract

“…POMDPs provide a rich framework for planning under both state transition uncertainty and observation uncertainty. The POMDP model has been widely used for asset management under uncertainty; see [14] and the references therein. Note that POMDPs are not well suited for machine maintenance, as they are based on a discrete time step: the unitary action taken at time t affects the state and reward at time t + 1.…”

Section: Abbreviations and Acronymsmentioning

confidence: 99%

Continuous-Observation Partially Observable Semi-Markov Decision Processes for Machine Maintenance

Zhang

Revie

2017

IEEE Trans. Rel.

View full text Add to dashboard Cite

Partially observable semi-Markov decision processes (POS-MDPs) provide a rich framework for planning under both state transition uncertainty and observation uncertainty. In this paper, we widen the literature on POSMDP by studying discretestate, discrete-action yet continuous-observation POSMDPs. We prove that the resultant α-vector set is continuous and therefore propose a point-based value iteration algorithm. This paper also bridges the gap between POSMDP and machine maintenance by incorporating various types of maintenance actions, such as actions changing machine state, actions changing degradation rate, and the temporally extended action "do nothing". Both finite and infinite planning horizons are reviewed, and the solution methodology for each type of planning horizon is given. We illustrate the maintenance decision process via a real industrial problem and demonstrate that the developed framework can be readily applied to solve relevant maintenance problems.

show abstract

Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation

Cited by 106 publications

References 29 publications

Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling

Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling

What makes long‐term monitoring convenient? A parametric analysis of value of information in infrastructure maintenance

Continuous-Observation Partially Observable Semi-Markov Decision Processes for Machine Maintenance

Contact Info

Product

Resources

About