A deep reinforcement learning framework for life-cycle maintenance planning of regional deteriorating bridges using inspection data

Lei, Xiaoming; Xia, Ye; Deng, Lu; Sun, Lijun

doi:10.1007/s00158-022-03210-3

Cited by 23 publications

(7 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For Equation (38), the value of pðs d js dÀ1 , aÞ is always set as a small value, such as \0.1. [62][63][64] Therefore, the order of magnitude rðs dÀ1 , aÞ, pðs d js dÀ1 , aÞa n + 1 ðs d Þ, and pðs dÀ1 js dÀ1 , aÞa n + 1 ðs dÀ1 Þ is much smaller than a n + 1 ðs d Þ. Other value coefficients a n ðs dÀi Þ can be demonstrated in a similar procedure.…”

Section: Declaration Of Conflicting Interestsmentioning

confidence: 82%

“…For Equation (38), the value of p ( s d | s d − 1 , a ) is always set as a small value, such as <0.1. 62–64 Therefore, the order of magnitude r ( s d − 1 , a ) , p ( s d | s d − 1 , a ) α n + 1 ( s d ) , and p ( s d − 1 | s d − 1 , a ) α n + 1 ( s d − 1 ) is much smaller than α n + 1 ( s d ) . Other value coefficients …”

Section: Figure 18mentioning

confidence: 99%

See 1 more Smart Citation

SHM-informed life-cycle intelligent maintenance of fatigue-sensitive detail using Bayesian forecasting and Markov decision process

Lai

Dong

Smyl

2023

Structural Health Monitoring

View full text Add to dashboard Cite

Civil and maritime engineering systems must be efficiently managed to control the failure risk at an acceptable level as their performance is gradually degraded throughout the operational life, caused by fatigue and corrosion. Structural health monitoring develops a timely capability to assess the structural condition and performance metrics. However, using actual long-term monitoring data to guide the life-cycle management under stochastic environments has not been sufficiently studied. To realize an optimal maintenance strategy within the service life, an integrated monitoring-based optimal management framework is developed on the basis of the partially observable Markov decision processes (POMDPs) and Bayesian forecasting. In the proposed framework, the stochastic fatigue processes are quantified by the state transition matrix. The Bayesian dynamic linear model is embedded in POMDPs as a continuous observation part to forecast the cycling impacts and estimate the deterioration rate using long-term dynamic strain responses. In addition, making use of the special features of the problem considered in this paper, an adaptive discretization strategy is proposed to alleviate the complexity of large discrete observed spaces in the POMDP. The applicability and feasibility of the framework are evaluated by intelligent maintenance of fatigue-sensitive components with real-world monitoring data. After solving the POMDP by an efficient offline solver, the results obtained in this paper demonstrate that structural interventions are uneconomical to extend the life when a welded detail is approaching its end of life due to the normal service. Furthermore, if multiple interventions are available, the framework can find optimal maintenance actions based on the trade-off between long-term utility and the corresponding cost. This framework as the prototype could also be adjusted to aid life-cycle intelligent maintenance of other types of components under different deterioration scenarios.

show abstract

Section: Declaration Of Conflicting Interestsmentioning

confidence: 82%

Section: Figure 18mentioning

confidence: 99%

SHM-informed life-cycle intelligent maintenance of fatigue-sensitive detail using Bayesian forecasting and Markov decision process

Lai

Dong

Smyl

2023

Structural Health Monitoring

View full text Add to dashboard Cite

show abstract

“…Xia et al (2021) developed a fully data-driven, regionallevel bridge maintenance management framework using data extracted from inspection reports [22]. Lei et al (2022) developed a framework for regional-level bridge maintenance planning based on deep reinforcement learning using data accumulated from years of bridge-inspection reports [23]. Informative data mining, on the other hand, seeks to identify patterns hidden inside massive data sets that a domain expert may not readily recognize.…”

Section: Introductionmentioning

confidence: 99%

A Big Data Approach for Investigating Bridge Deterioration and Maintenance Strategies in Taiwan

2023

View full text Add to dashboard Cite

Due to the dwindling maintenance budget and lack of qualified bridge inspectors, bridge-management agencies in Taiwan need to develop cost-effective maintenance and inspection strategies to preserve the safety and functionality of their aging, natural disaster-prone bridges. To inform the development of such a strategy, this study examined the big data stored in the Taiwan Bridge Management System (TBMS) using the knowledge discovery in databases (KDD) process. Cluster and association algorithms were applied to the inventory and five-year inspection data of 2849 bridges to determine the bridge structural configurations and components that are prone to deterioration. Bridge maintenance agencies can use the results presented to reevaluate their current maintenance and inspection strategies and concentrate their limited resources on bridges and components most prone to deterioration.

show abstract

“…This paper considers optimal planning for a class of structured budget-constrained POMDPs. This setting is motivated by infrastructure maintenance planning -a widely studied problem [9,10,11] that involves finding the optimal policy for maintenance and inspection, of an infrastructure component or a group of components within a certain budget [12,13]. For simplicity of planning, the stochastic dynamics of multi-component systems are modelled using a POMDP [14,15].…”

Section: Introductionmentioning

confidence: 99%

“…where the inequality is due to the assumption that relation (10) holds for horizon H. Also, using (12) we know that:…”

mentioning

confidence: 99%

Welfare Maximization Algorithm for Solving Budget-Constrained Multi-Component POMDPs

Vora¹,

Thangeda²,

Grussing³

et al. 2023

Preprint

View full text Add to dashboard Cite

Partially Observable Markov Decision Processes (POMDPs) provide an efficient way to model realworld sequential decision making processes. Motivated by the problem of maintenance and inspection of a group of infrastructure components with independent dynamics, this paper presents an algorithm to find the optimal policy for a multi-component budget-constrained POMDP. We first introduce a budgeted-POMDP model (b-POMDP) which enables us to find the optimal policy for a POMDP while adhering to budget constraints. Next, we prove that the value function or maximal collected reward for a b-POMDP is a concave function of the budget for the finite horizon case. Our second contribution is an algorithm to calculate the optimal policy for a multi-component budget-constrained POMDP by finding the optimal budget split among the individual component POMDPs. The optimal budget split is posed as a welfare maximization problem and the solution is computed by exploiting the concave nature of the value function. We illustrate the effectiveness of the proposed algorithm by proposing a maintenance and inspection policy for a group of real-world infrastructure components with different deterioration dynamics, inspection and maintenance costs. We show that the proposed algorithm vastly outperforms the policy currently used in practice.

show abstract

A deep reinforcement learning framework for life-cycle maintenance planning of regional deteriorating bridges using inspection data

Cited by 23 publications

References 50 publications

SHM-informed life-cycle intelligent maintenance of fatigue-sensitive detail using Bayesian forecasting and Markov decision process

SHM-informed life-cycle intelligent maintenance of fatigue-sensitive detail using Bayesian forecasting and Markov decision process

A Big Data Approach for Investigating Bridge Deterioration and Maintenance Strategies in Taiwan

Welfare Maximization Algorithm for Solving Budget-Constrained Multi-Component POMDPs

Contact Info

Product

Resources

About