Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Shuai, Bin; Zhou, Quan; Li, Ji; He, Yinglong; Li, Ziyang; Williams, Howard; Xu, Hongming; Shuai, Shijin

doi:10.1016/j.apenergy.2020.114900

Cited by 50 publications

(29 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where batt is the power supplied by the battery pack and egu is the power supplied by the engine generator; egu_max is the maximum power that can be provided by the engine-generator; and dem is the vehicle's power demand. Energy management strategies, including exponential functions [7], model-based predictive control [9], [10], and model-free control [13], [14], build a nonlinear relationship between the power distribution and the vehicle states (e.g. battery SoC).…”

Section: ) Energy Management Modulementioning

confidence: 99%

See 1 more Smart Citation

Modified Particle Swarm Optimization With Chaotic Attraction Strategy for Modular Design of Hybrid Powertrains

Zhou

Zhao

et al. 2021

IEEE Trans. Transp. Electrific.

Self Cite

View full text Add to dashboard Cite

Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document.When citing, please reference the published version. Take down policyWhile the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has been uploaded in error or has been deemed to be commercially or otherwise sensitive.

show abstract

Section: ) Energy Management Modulementioning

confidence: 99%

“…( 13) and Eq. (14). Logistic map, which follows a principle of biological evidencing behavior, is proposed as a unified chaotic attraction strategy in this paper because it has only one tuning parameter and is easy to be implemented in engineering applications [47].…”

Section: Chaotic Attraction With Logistic Mapmentioning

confidence: 99%

Modified Particle Swarm Optimization With Chaotic Attraction Strategy for Modular Design of Hybrid Powertrains

Zhou

Zhao

et al. 2021

IEEE Trans. Transp. Electrific.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The vehicle speed dataset is collected from the real driving conditions. To reduce the training time and improve the control accuracy, the vehicle velocity is divided into three-speed intervals that are [0-12] m/s, [12][13][14][15][16][17][18][19][20][21][22][23][24] m/s, [24][25][26][27][28][29][30][31][32][33][34][35][36] m/s, and they represent low, medium, and high speed, respectively. Then, the classified speed intervals are adopted to train the DDPG algorithm separately until the algorithm converges, the trained neural network is stored, as depicted in Fig.…”

Section: A a Bi-level Frameworkmentioning

confidence: 99%

“…In Ref. [30], B Shuai. et al employed the max-value-based policy and the random policy to reduce the overestimation for double Q-learning.…”

Section: Introductionmentioning

confidence: 99%

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Guo

Tang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL). This work aims to address the defect of DRL in tedious training time. First, an optimization control modeling of a hybrid tracked vehicle is built, wherein the elaborate powertrain components are introduced. Then, a bi-level control framework is constructed to derive the energy management strategies (EMSs). The upper-level is applying the particular deep deterministic policy gradient (DDPG) algorithms for EMS training at different speed intervals. The lower-level is employing the TL method to transform the pre-trained neural networks for a novel driving cycle. Finally, a series of experiments are executed to prove the effectiveness of the presented control framework. The optimality and adaptability of the formulated EMS are illuminated. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.

show abstract

“…The effectiveness of RL has been demonstrated in various vehiclecontrol-related applications [34]. Remarkable improvements in vehicle energy efficiency have been achieved by RL methods, e.g., Q-learning [35], deep Q-learning [36], double Q-learning [37], and multiple-step Q-learning [38]. Most research on RL-based power management control focus on learning from scratch [39], [40].…”

mentioning

confidence: 99%

Knowledge Implementation and Transfer With an Adaptive Learning Network for Real-Time Power Management of the Plug-in Hybrid Vehicle

Zhou

Zhao

Shuai

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

Essential decision-making tasks such as power management in future vehicles will benefit from the development of artificial intelligence technology for safe and energy-efficient operations. To develop the technique of using neural network and deep learning in energy management of the plug-in hybrid vehicle and evaluate its advantage, this article proposes a new adaptive learning network that incorporates a deep deterministic policy gradient (DDPG) network with an adaptive neuro-fuzzy inference system (ANFIS) network. First, the ANFIS network is built using a new global K-fold fuzzy learning (GKFL) method for real-time implementation of the offline dynamic programming result. Then, the DDPG network is developed to regulate the input of the ANFIS network with the real-world reinforcement signal. The ANFIS and DDPG networks are integrated to maximize the control utility (CU), which is a function of the vehicle's energy efficiency and the battery state-of-charge. Experimental studies are conducted to testify the performance and robustness of the DDPG-ANFIS network. It has shown that the studied vehicle with the DDPG-ANFIS network achieves 8% higher CU than using the MATLAB ANFIS toolbox on the studied vehicle. In five simulated real-world driving conditions, the DDPG-ANFIS network increased the maximum mean CU value by 138% over the ANFIS-only network and 5% over the DDPG-only network.

show abstract

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Cited by 50 publications

References 43 publications

Modified Particle Swarm Optimization With Chaotic Attraction Strategy for Modular Design of Hybrid Powertrains

Modified Particle Swarm Optimization With Chaotic Attraction Strategy for Modular Design of Hybrid Powertrains

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Knowledge Implementation and Transfer With an Adaptive Learning Network for Real-Time Power Management of the Plug-in Hybrid Vehicle

Contact Info

Product

Resources

About