Driving Torque Distribution Strategy of Skid-Steering Vehicles with Knowledge-Assisted Reinforcement Learning

Abstract

Due to the advantages of their drive configuration form, skid-steering vehicles with independent wheel drive systems are widely used in various special applications. However, obtaining a reasonable distribution of the driving torques for the coordinated control of independent driving wheels is a challenging problem. In this paper, we propose a torque distribution strategy based on the Knowledge-Assisted Deep Deterministic Policy Gradient (KA-DDPG) algorithm, in order to minimize the desired value tracking error as well as achieve the longitudinal speed and yaw rate tracking control of skid-steering vehicles. The KA-DDPG algorithm combines knowledge-assisted learning methods with the DDPG algorithm, within the framework of knowledge-assisted reinforcement learning. To accelerate the learning process of KA-DDPG, two assisted learning methods are proposed: a criteria action method and a guiding reward method. The simulation results obtained, considering different scenarios, demonstrate that the KA-DDPG-based torque distribution strategy allows a skid-steering vehicle to achieve high performance, in tracking the desired value. In addition, further simulation results, also, demonstrate the contributions of knowledge-assisted learning methods to the training process of KA-DDPG: the criteria action method speeds up the learning speed by reducing the agent’s random action selection, while the guiding reward method achieves the same result by sharpening the reward function.

MOGA 4WD: multi-objective genetic algorithm for four-wheel drive electrical vehicle torque distribution in challenging conditions

“…However, the friction forces are proportional to the torques applied on the wheels. Equation (1) shows the relationship between applied forces and moments with geometric parameters of the vehicle and the terrain. In this problem, there are four unknown variables and three equations.…”

“…The application of mobile robots or off-road vehicles for navigation on uneven terrain has grown in recent years, especially in applications involving agriculture, mining, or military purposes [1]. In order to optimize navigation in this type of terrain, several solutions aim to optimize the geometry [2][3][4] of the systems or their control strategies [5][6][7].…”

“…In this work, a skid-steer locomotion system is applied. Although limited for use in terrains with large obstacles or narrow gaps, this type of system has a high power-toweight ratio and good maneuverability, combined with low investment costs [1,5,8].…”

Dynamic Goal Tracking for Differential Drive Robot using Deep Reinforcement Learning

“…Extensively used on unmanned aerial vehicles (UAVs) [10], covering legged robots of OpenAI Gym environments including ant, half cheetah and walker [8]. Moreover, fewer implementations of DDPG on differential drives are available in literature, e.g., autonomous driving cars [11], skid steering in differential drive [12], optimal torque distribution [13], and obstacle detection for differential drive [14]. However, implementation of TD3 on differential drive to the best of our knowledge has not been done.…”