Active Steering Controller for Driven Independently Rotating Wheelset Vehicles Based on Deep Reinforcement Learning

Abstract

This paper proposes an active steering controller for Driven Independently Rotating Wheelset (DIRW) vehicles based on deep reinforcement learning (DRL). For the two-axle railway vehicles equipped with Independently Rotating Wheelsets (IRWs), each wheel connected to a wheel-side motor, the Ape-X DDPG controller, an enhanced version of the Deep Deterministic Policy Gradient (DDPG) algorithm, is adopted. Incorporating Distributed Prioritized Experience Replay (DPER), Ape-X DDPG trains neural network function approximators to obtain a data-driven DIRW active steering controller. This controller is utilized to control the input torque of each wheel, aiming to improve the steering capability of IRWs. Simulation results indicate that compared to the existing model-based H∞ control algorithm and data-driven DDPG control algorithm, the Ape-X DDPG active steering controller demonstrates better curving steering performance and centering ability in straight tracks across different running conditions and significantly reduces wheel–rail wear. To validate the proposed algorithm’s efficacy in real vehicles, a 1:5 scale model of the DIRW vehicle and its digital twin dynamic model were designed and manufactured. The proposed control algorithm was deployed on the scale vehicle and subjected to active steering control experiments on a scaled track. The experimental results reveal that under the active steering control of the Ape-X DDPG controller, the steering performance of the DIRW scale model on both straight and curved tracks is significantly enhanced.

Multiagent Reinforcement Learning for Active Guidance Control of Railway Vehicles with Independently Rotating Wheels

“…For a given experience tuple j, agent i calculates the current Q value dependent on the s j and the collective actions, while the target critic network determines the Q value for s j+1 . The TD error can be formulated using Equation (13).…”

“…Recognizing the limitations of traditional control strategies, there is a growing interest in data-driven methods. Our previous research explored the application of deep reinforcement learning (DRL)-based controllers, including the deep deterministic policy gradient (DDPG) and Ape-X DDPG [13,14], leveraging deep neural networks' ability to fit nonlinear systems. Nevertheless, several limitations are encountered: (a) the existing DRL-based controllers require multiple dynamic parameters from all IRWs, and the high dimensionality of observation spaces leads to slow convergence during training; (b) current strategies mainly focus on the centralized control of the entire vehicle without achieving local control for an individual IRW, potentially affecting computational efficiency in practical applications.…”