Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle

Zhou, Quan; Ji, Li; Shuai, Bin; Williams, Howard; He, Yinglong; Li, Ziyang; Xu, Hongming; Yan, Fuwu

doi:10.1016/j.apenergy.2019.113755

Cited by 116 publications

(52 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The vehicle speed dataset is collected from the real driving conditions. To reduce the training time and improve the control accuracy, the vehicle velocity is divided into three-speed intervals that are [0-12] m/s, [12][13][14][15][16][17][18][19][20][21][22][23][24] m/s, [24][25][26][27][28][29][30][31][32][33][34][35][36] m/s, and they represent low, medium, and high speed, respectively. Then, the classified speed intervals are adopted to train the DDPG algorithm separately until the algorithm converges, the trained neural network is stored, as depicted in Fig.…”

Section: A a Bi-level Frameworkmentioning

confidence: 99%

“…In [23], better power distributions between the battery and the ultracapacitor of PHEVs were obtained through the RL-based method, and 16.8% energy loss reduction was achieved. The researchers in [24] and [25] adopted the predicted EMS to improve the HEVs' performance. However, the discrete state space and action space of RL hinder its further application in EMS of HEVs, and the emergence of deep reinforcement learning (DRL) has bridged over this difficulty.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Guo

Tang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL). This work aims to address the defect of DRL in tedious training time. First, an optimization control modeling of a hybrid tracked vehicle is built, wherein the elaborate powertrain components are introduced. Then, a bi-level control framework is constructed to derive the energy management strategies (EMSs). The upper-level is applying the particular deep deterministic policy gradient (DDPG) algorithms for EMS training at different speed intervals. The lower-level is employing the TL method to transform the pre-trained neural networks for a novel driving cycle. Finally, a series of experiments are executed to prove the effectiveness of the presented control framework. The optimality and adaptability of the formulated EMS are illuminated. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.

show abstract

Section: A a Bi-level Frameworkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Guo

Tang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…PHEVs are widely promoted as an efficient and clean solution that combines an internal combustion engine (ICE) with an electric motor and a large rechargeable battery. This hybrid powertrain enables all-electric driving for extended periods of time and overcomes the concern of range anxiety [5], [6]. Intensive efforts on PHEVs have developed energy management strategies (EMS) for coordinating the power split in a fuel-efficient way [7].…”

Section: Introductionmentioning

confidence: 99%

Multiobjective Co-Optimization of Cooperative Adaptive Cruise Control and Energy Management Strategy for PHEVs

Zhou

Makridis

et al. 2020

IEEE Trans. Transp. Electrific.

Self Cite

View full text Add to dashboard Cite

Electrification, automation, and connectivity in the automotive and transport industries are gathering momentum, but there are escalating concerns over their need for co-optimization to improve energy efficiency, traffic safety, and ride comfort. Previous approaches to these multi-objective co-optimization problems often overlook trade-offs and scale differences between the objectives, resulting in misleading optimizations. To overcome these limitations, this study proposes a Pareto-based framework that demonstrably optimizes the system parameters of the cooperative adaptive cruise control (CACC) and the energy management strategy (EMS) for PHEVs. The high-level Pareto knowledge assists in finding a best-compromise solution. The results of this work suggest that the energy and the comfort targets are harmonious, but both conflict with the safety target. Validation using real-world driving data shows that the Pareto optimum for CACC and EMS systems, relative to the baseline, can reduce energy consumption (by 7.57 %) and tracking error (by 68.94 %), while simultaneously satisfying ride comfort needs. In contrast to the weighted-sum method, the proposed Pareto method can optimally balance and scale the multiple objective functions. In addition, sensitivity analysis proves that the vehicle reaction time impacts significantly on tracking safety, but its effect on energy saving is trivial.

show abstract

“…Liu et al implement Q-learning algorithms for energy management of series hybrid vehicles [32]. Zhou et al proposed multi-step reinforcement learning for energy management of a hybrid vehicle [33]. Cao et al optimize the energy use of a plug-in hybrid vehicle based on Q-learning [34].…”

Section: Introductionmentioning

confidence: 99%

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Shuai

Zhou

et al. 2020

Applied Energy

Self Cite

View full text Add to dashboard Cite

Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document. When citing, please reference the published version. Take down policy While the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has been uploaded in error or has been deemed to be commercially or otherwise sensitive.

show abstract

Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle

Cited by 116 publications

References 37 publications

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Multiobjective Co-Optimization of Cooperative Adaptive Cruise Control and Energy Management Strategy for PHEVs

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Contact Info

Product

Resources

About