Optimizing Subway Train Operation With Hierarchical Adaptive Control Approach

Abstract

The proportional integral derivative (PID) method is widely used in industrial control applications. However, when applied to complex and dynamic train operation control systems, real-time parameter adjustment becomes a formidable challenge. Moreover, the multifaceted nature of train operation control, encompassing safety, parking precision, passenger comfort, and energy efficiency, exacerbates the difficulty of parameter adjustment. To address this problem, this paper formulates train operation control as a Markov decision process (MDP) and introduces an innovative adaptive control approach. This approach features a hierarchical structure comprising an upper-level deep deterministic policy gradient (DDPG) controller and a lower-level PID controller, leveraging the learning capability of the DDPG algorithm, as well as the stability and interpretability of the PID method. The upper-level controller acquires train status information and autonomously fine-tunes the PID parameters, while the lower-level controller accepts these parameters and adjusts the percentage of traction or braking to achieve train operation control. Furthermore, the reward function has been meticulously designed to reconcile the diverse objectives of train operation. Extensive experiments conducted on a subway simulation platform substantiate the effectiveness and adaptability of the proposed approach in various operational scenarios.INDEX TERMS subway train operation, reinforcement learning, adaptive control, reward function.