Condition-Based Maintenance with Reinforcement Learning for Dry Gas Pipeline Subject to Internal Corrosion

Abstract

Gas pipeline systems are one of the largest energy infrastructures in the world and are known to be very efficient and reliable. However, this does not mean they are prone to no risk. Corrosion is a significant problem in gas pipelines that imposes large risks such as ruptures and leakage to the environment and the pipeline system. Therefore, various maintenance actions are performed routinely to ensure the integrity of the pipelines. The costs of the corrosion-related maintenance actions are a significant portion of the pipeline’s operation and maintenance costs, and minimizing this large cost is a highly compelling subject that has been addressed by many studies. In this paper, we investigate the benefits of applying reinforcement learning (RL) techniques to the corrosion-related maintenance management of dry gas pipelines. We first address the rising need for a simulated testbed by proposing a test bench that models corrosion degradation while interacting with the maintenance decision-maker within the RL environment. Second, we propose a condition-based maintenance management approach that leverages a data-driven RL decision-making methodology. An RL maintenance scheduler is applied to the proposed test bench, and the results show that applying the proposed condition-based maintenance management technique can reduce up to 58% of the maintenance costs compared to a periodic maintenance policy while securing pipeline reliability.

System-level prognosis and health monitoring modeling framework and software implementation for gas pipeline system integrity management

“…The optimal maintenance schedules are costeffective. When comparing the monthly average maintenance cost with 12 normal (periodic) maintenance costs retrieved from the literature (Mahmoodzadeh et al, 2020), the difference is noticeable as shown in Figure 58 and Table 14. As shown in Table 14, the optimization of the operating parameters aims at: (i) extending the lifetime of the pipeline, (ii) minimizing the maintenance cost, (iii) maximizing the economic profit, and (iv) minimizing the CO2 emission amount.…”

“…Maintenance Activities and their related Cost and Downtime. The presented data are extracted from the literature (Mahmoodzadeh et al, 2020). The last step in the optimization model is to define the constraints.…”

“…The threshold values of the failure probability together with the corresponding recommended practices are shown in Table 3. (Mahmoodzadeh et al, 2020). The goal of a RL-based agent is to maximize expected cumulative rewards in the long run.…”

“…After the agent is well trained, the Q-table representing updated Q values with respect to all state spaces of the pipeline system is used in this dissertation to provide the optimized maintenance schedule. For more details about the development of the maintenance scheduler, the readers are referred to (Mahmoodzadeh et al, 2020).…”

On risk-based maintenance: A comprehensive review of three approaches to track the impact of consequence modelling for predicting maintenance actions

“…While time-based PM schedules maintenance actions at fixed time-intervals, CBM reacts based on available information regarding the condition of a given component [9]. Due to its advantages many researchers have recently focused on CBM application to several fields [10][11][12][13][14].…”