Using reinforcement learning (RL) for torque distribution of skid steering vehicles has attracted increasing attention recently. Various RL-based torque distribution methods have been proposed to deal with this classical vehicle control problem, achieving a better performance than traditional control methods. However, most RL-based methods focus only on improving the performance of skid steering vehicles, while actuator faults that may lead to unsafe conditions or catastrophic events are frequently omitted in existing control schemes. This study proposes a meta-RL-based fault-tolerant control (FTC) method to improve the tracking performance of vehicles in the case of actuator faults. Based on meta deep deterministic policy gradient (meta-DDPG), the proposed FTC method has a representative gradient-based metalearning algorithm workflow, which includes an offline stage and an online stage. In the offline stage, an experience replay buffer with various actuator faults is constructed to provide data for training the metatraining model; then, the metatrained model is used to develop an online meta-RL update method to quickly adapt its control policy to actuator fault conditions. Simulations of four scenarios demonstrate that the proposed FTC method can achieve a high performance and adapt to actuator fault conditions stably.
Due to the advantages of their drive configuration form, skid-steering vehicles with independent wheel drive systems are widely used in various special applications. However, obtaining a reasonable distribution of the driving torques for the coordinated control of independent driving wheels is a challenging problem. In this paper, we propose a torque distribution strategy based on the Knowledge-Assisted Deep Deterministic Policy Gradient (KA-DDPG) algorithm, in order to minimize the desired value tracking error as well as achieve the longitudinal speed and yaw rate tracking control of skid-steering vehicles. The KA-DDPG algorithm combines knowledge-assisted learning methods with the DDPG algorithm, within the framework of knowledge-assisted reinforcement learning. To accelerate the learning process of KA-DDPG, two assisted learning methods are proposed: a criteria action method and a guiding reward method. The simulation results obtained, considering different scenarios, demonstrate that the KA-DDPG-based torque distribution strategy allows a skid-steering vehicle to achieve high performance, in tracking the desired value. In addition, further simulation results, also, demonstrate the contributions of knowledge-assisted learning methods to the training process of KA-DDPG: the criteria action method speeds up the learning speed by reducing the agent’s random action selection, while the guiding reward method achieves the same result by sharpening the reward function.
Meta-reinforcement learning (meta-RL), used in the fault-tolerant control (FTC) problem, learns a meta-trained model from a set of fault situations that have a high-level similarity. However, in the real world, skid-steering vehicles might experience different types of fault situations. The use of a single initial meta-trained model limits the ability to learn different types of fault situations that do not possess a strong similarity. In this paper, we propose a novel FTC method to mitigate this limitation, by meta-training multiple initial meta-trained models and selecting the most suitable model to adapt to the fault situation. The proposed FTC method is based on the meta deep deterministic policy gradient (meta-DDPG) algorithm, which includes an offline stage and an online stage. In the offline stage, we first train multiple meta-trained models corresponding to different types of fault situations, and then a situation embedding model is trained with the state-transition data generated from meta-trained models. In the online stage, the most suitable meta-trained model is selected to adapt to the current fault situation. The simulation results demonstrate that the proposed FTC method allows skid-steering vehicles to adapt to different types of fault situations stably, while requiring significantly fewer fine-tuning steps than the baseline.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.