Fault-Tolerant Control of Degrading Systems with On-Policy Reinforcement Learning

Ahmed, Ibrahim; Quiñones-Grueiro, Marcos; Biswas, Gautam

doi:10.1016/j.ifacol.2020.12.878

Cited by 11 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithm was evaluated on a simulation of a fuel transfer system of an aircraft. The system is defined in greater detail in (Ahmed et al, 2020). The objective is to maintain center of gravity, variance in fuel distribution, and closed valves to avoid unnecessary mass transfer.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Complementary Meta-Reinforcement Learning for Fault-Adaptive Control

Ahmed,

Quinones-Grueiro,

Biswas

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Faults are endemic to all systems. Adaptive fault-tolerant control maintains degraded performance when faults occur as opposed to unsafe conditions or catastrophic events. In systems with abrupt faults and strict time constraints, it is imperative for control to adapt quickly to system changes to maintain system operations. We present a meta-reinforcement learning approach that quickly adapts its control policy to changing conditions. The approach builds upon modelagnostic meta learning (MAML). The controller maintains a complement of prior policies learned under system faults. This "library" is evaluated on a system after a new fault to initialize the new policy. This contrasts with MAML, where the controller derives intermediate policies anew, sampled from a distribution of similar systems, to initialize a new policy. Our approach improves sample efficiency of the reinforcement learning process. We evaluate our approach on an aircraft fuel transfer system under abrupt faults.

show abstract

Section: Methodsmentioning

confidence: 99%

“…In our past work (Ahmed, Quiones-Grueiro, & Biswas, 2020), we developed data-driven models to supplement experience with the real environment and simulate faults. In this work, we employ meta-RL for faster adaption of the RL algorithm to collected data samples.…”

Section: Introductionmentioning

confidence: 99%

Complementary Meta-Reinforcement Learning for Fault-Adaptive Control

Ahmed,

Quinones-Grueiro,

Biswas

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many researchers have applied this feature to quickly adapt to system failures and external disturbances. In [ 20 , 21 , 22 , 23 ], the authors presented a series of methods based on meta-RL to quickly adapt their control policies to maintain degraded performance when faults occur in the aircraft fuel transfer system. The scheme of FTC methods includes offline metatraining and online metatesting stages.…”

Section: Related Workmentioning

confidence: 99%

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Dai

Chen

Yang

2022

Sensors

View full text Add to dashboard Cite

Using reinforcement learning (RL) for torque distribution of skid steering vehicles has attracted increasing attention recently. Various RL-based torque distribution methods have been proposed to deal with this classical vehicle control problem, achieving a better performance than traditional control methods. However, most RL-based methods focus only on improving the performance of skid steering vehicles, while actuator faults that may lead to unsafe conditions or catastrophic events are frequently omitted in existing control schemes. This study proposes a meta-RL-based fault-tolerant control (FTC) method to improve the tracking performance of vehicles in the case of actuator faults. Based on meta deep deterministic policy gradient (meta-DDPG), the proposed FTC method has a representative gradient-based metalearning algorithm workflow, which includes an offline stage and an online stage. In the offline stage, an experience replay buffer with various actuator faults is constructed to provide data for training the metatraining model; then, the metatrained model is used to develop an online meta-RL update method to quickly adapt its control policy to actuator fault conditions. Simulations of four scenarios demonstrate that the proposed FTC method can achieve a high performance and adapt to actuator fault conditions stably.

show abstract

“…In ref , the widespread application prospect of RL in solving optimization and control problems is elaborated. It is adaptable to discrete, , continuous, − normal, and faulty systems. , A new strategy iterative algorithm was proposed for optimizing regulator design without the need for a system model . An FTC method based on RL was designed for unknown affine nonlinear systems with actuator faults, which handled stuck faults without understanding the system dynamics.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning-Based Optimal Fault-Tolerant Tracking Control of Industrial Processes

Wang,

Li,

Zhang

et al. 2023

Ind. Eng. Chem. Res.

View full text Add to dashboard Cite

Fault-Tolerant Control of Degrading Systems with On-Policy Reinforcement Learning

Cited by 11 publications

References 19 publications

Complementary Meta-Reinforcement Learning for Fault-Adaptive Control

Complementary Meta-Reinforcement Learning for Fault-Adaptive Control

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Reinforcement Learning-Based Optimal Fault-Tolerant Tracking Control of Industrial Processes

Contact Info

Product

Resources

About