Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control

Ahmed, Ibrahim; Khorasgani, Hamed; Biswas, Gautam

doi:10.1016/j.ifacol.2018.09.583

Cited by 15 publications

(7 citation statements)

References 15 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The history of prior states does not affect the value or reward of following states and actions. This is known as the Markov property [7]. In the most exciting and challenging cases, actions may affect not only the immediate reward but also the following situation and, through that, all subsequent rewards.…”

Section: Reinforcement Learning (Rl) Methodsmentioning

confidence: 99%

A Q-Learning based Fault-Tolerant Controller with Application to CSTH System

Hosseini¹,

Salahshoor²

2021

Software Engineering and Applications

View full text Add to dashboard Cite

Systems are continually subjected to faults or malfunctions because of age or sudden events, which might degrade the operation performance and even result in operation failure that is a quite important issue in safety-critical systems. Thus, this important problem is the main reason to use the Fault-Tolerant strategy to improve the system’s performance with the presence of faults. A fascinating property in Fault-Tolerant Controllers (FTCs) is adaptability to system changes as they evolve throughout system operations. In this paper, a Q-learning algorithm with a greedy policy was used to realize the FTC adaptability. Then, some fault scenarios are introduced in a Continuous Stirred Tank Heater (CSTH) to compare the closed-loop performance of the developed Q-learning-based FTC with concerning conventional PID controller and an RL-based FTC. The obtained results show the effectiveness of Q-learningbased FTC in different fault scenarios.

show abstract

Section: Reinforcement Learning (Rl) Methodsmentioning

confidence: 99%

A Q-Learning based Fault-Tolerant Controller with Application to CSTH System

Hosseini¹,

Salahshoor²

2021

Software Engineering and Applications

View full text Add to dashboard Cite

show abstract

“…Many researchers have applied this feature to quickly adapt to system failures and external disturbances. In [ 20 , 21 , 22 , 23 ], the authors presented a series of methods based on meta-RL to quickly adapt their control policies to maintain degraded performance when faults occur in the aircraft fuel transfer system. The scheme of FTC methods includes offline metatraining and online metatesting stages.…”

Section: Related Workmentioning

confidence: 99%

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Dai

Chen

Yang

2022

Sensors

View full text Add to dashboard Cite

Using reinforcement learning (RL) for torque distribution of skid steering vehicles has attracted increasing attention recently. Various RL-based torque distribution methods have been proposed to deal with this classical vehicle control problem, achieving a better performance than traditional control methods. However, most RL-based methods focus only on improving the performance of skid steering vehicles, while actuator faults that may lead to unsafe conditions or catastrophic events are frequently omitted in existing control schemes. This study proposes a meta-RL-based fault-tolerant control (FTC) method to improve the tracking performance of vehicles in the case of actuator faults. Based on meta deep deterministic policy gradient (meta-DDPG), the proposed FTC method has a representative gradient-based metalearning algorithm workflow, which includes an offline stage and an online stage. In the offline stage, an experience replay buffer with various actuator faults is constructed to provide data for training the metatraining model; then, the metatrained model is used to develop an online meta-RL update method to quickly adapt its control policy to actuator fault conditions. Simulations of four scenarios demonstrate that the proposed FTC method can achieve a high performance and adapt to actuator fault conditions stably.

show abstract

“…To verify the control effect of the designed sliding mode controller on the non-linear system, a simulation test is carried out for the rotational speed of the high-pressure rotor. Firstly, through debugging, the parameters p=7 ， q=5 ， = 2 ， = 2.5， =1,2 are set in formula (9); and in the exponential approaching law = 5， = 10; in the low-pass filter = 20. Besides, since the above-mentioned controller is applied to the outer loop in the large closed-loop rotational speed of aero-engine and the output of the outer loop is the given position Lr of the metering valve, the given fuel quantity needs to be converted into Lr and then provided to the small closedloop actuator position.…”

Section: ̇= − ⋅ ( ) − ⋅ (22)mentioning

confidence: 99%

“…For example, Wallhagen et al [7] performed fault diagnosis by comparing the response difference of the engine output between step input and normal control, and on this basis, the fault sensor signal is reconstructed by the resolution margin to realize the fault tolerance of the rotational speed closed-loop control loop; Napolitan et al [8] proposed a sensor fault diagnosis and faulttolerant control method based on online BP(Back Propagation) neural network. In recent years, Ahmed et al [9] designed an adaptive fault-tolerant controller based on the C-130 aircraft fuel tank model; Chen, L et al [10] developed a fault-tolerant integral sliding mode control scheme based on a large civil Boeing 747 engine model, which is through effective use of actuator redundancy to retain the nominal (fault-free) closedloop performance when the actuator fails or faults. Seo et al [11] proposed a design method of single-engined aircraft faulttolerant control system to deal with the problem of the gradual loss of reasoning over time.…”

Section: Introductionmentioning

confidence: 99%

Intelligent Fault-Tolerant Control System Design and Semi-Physical Simulation Validation of Aero-Engine

Liu¹,

Chen

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In order to improve the reliability and real-time of the control system of aero-engine, an intelligent fault-tolerant control system based on the online sequential extreme learning machine (OS-ELM) is proposed against the sensor faults. This system can realize the online fault diagnosis and signal reconstruction without a system model. And while considering the traditional PID control robustness and poor anti-interference ability and other shortcomings, an improved global fast non-singular terminal sliding mode control method is used to obtain better control effects, effectively solve the uncertainty problem in aero-engine, and give full play to aero-engine performance. To verify the feasibility and effectiveness of this system based on the above method, a turbofan engine is taken as the research object and semi-physical simulation experiments on fault-tolerant control are conducted on a semi-physical simulation test platform. Results show that the controller of this system can safely and reliably control the aero-engine without losing its control performance under the circumstance that there are faults in engine sensors. The purpose of faulttolerant control is reached.

show abstract

Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control

Cited by 15 publications

References 15 publications

A Q-Learning based Fault-Tolerant Controller with Application to CSTH System

A Q-Learning based Fault-Tolerant Controller with Application to CSTH System

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Intelligent Fault-Tolerant Control System Design and Semi-Physical Simulation Validation of Aero-Engine

Contact Info

Product

Resources

About