A deep reinforcement learning framework for optimizing fuel economy of hybrid electric vehicles

Zhao, Pu; Wang, Yanzhi; Chang, Naehyuck; Zhu, Qi; Lin, Xue

doi:10.1109/aspdac.2018.8297305

Cited by 33 publications

(22 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, more data are required in hybrid-algorithm cases. In [26], the authors applied a deep neural network (DNN) to train the offline value functions and used the Qlearning algorithm to compute the online controls, which can be adaptive to different powertrain modeling and driving situations. The authors of [27] constructed DRL-enabled energy-management strategies that considered different drivers' behaviors, which could improve fuel efficiency.…”

Section: Hybrid Algorithmsmentioning

confidence: 99%

Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects

et al. 2019

EEE Ind. Electron. Mag.

191

View full text Add to dashboard Cite

E nergy management is a critical technology in plug-in hybrid-electric vehicles (PHEVs) for maximizing efficiency, fuel economy, and range, as well as reducing pollutant emissions. At the same time, deep reinforcement learning (DRL) has become an effective and important methodology to formulate model-free and realtime energy-management strategies for HEVs and PHEVs. In this article, we describe the energy-management issues of HEVs/PHEVs and summarize a variety of potential DRL applications for onboard energy management. We also discuss the prospects for various DRL approaches in the energy-management field.

show abstract

Section: Hybrid Algorithmsmentioning

confidence: 99%

Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects

et al. 2019

EEE Ind. Electron. Mag.

191

View full text Add to dashboard Cite

show abstract

“…Based on the DQL method and considered as a breakthrough in the field of reinforcement learning, a deep reinforcement learning (DRL) framework for optimizing the fuel economy of HEVs was put forward in [99], which is approximately optimal, model-free, and has no need to have any prior knowledge of the driving cycle. The DRL technique consists of an offline deep neural network and an online deep Q-learning network, and can handle the high dimensional state and action space.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…This strategy can have a good performance in limiting the maximum discharge current and optimizing the system efficiency. To seek the optimal control, different forgetting factors and Kullback-Leibler (KL) [97,99,101] divergence rates, which decide whether to update the power management strategy, were discussed.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Towards a Smarter Energy Management System for Hybrid Vehicles: A Comprehensive Review of Control Strategies

Kong

Liang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

This paper presents a comprehensive review of energy management control strategies utilized in hybrid electric vehicles (HEVs). These can be categorized as rule-based strategies and optimization-based strategies. Rule-based strategies, as the most basic strategy, are widely used due to their simplicity and practical application. The focus of rule-based strategies is to determine and optimize the optimal threshold for mode switching; however, they fall into a local optimal solutions. To have better performance in energy management, optimization-based strategies were developed. The categories of the existing optimization-based strategies are identified from the latest literature, and a brief study of each strategy is discussed, which consists of the main research ideas, the research focus, advantages, disadvantages and improvements to ameliorate optimality and real-time performance. Deterministic dynamic programming strategy is regarded as a benchmark. Based on neural network and the large data processing technology, data-driven strategies are put forward due to their approximate optimality and high computational efficiency. Finally, the comprehensive performance of each control strategy is analyzed with respect to five aspects. This paper not only provides a comprehensive analysis of energy management control strategies for HEVs, but also presents the emphasis in the future.

show abstract

“…In [20], the weighted sum of the fuel and battery electric energy use is defined as the cost function of the Q-learning algorithm, which is then applied to a 48V mild HEV. More recent techniques such as Deep Q Networks are utilized in [21]- [24], which combine Q-learning with a deep neural network to obtain fast convergence and improve the learning performance; similarly, gradient-based methods such as Deep Deterministic Policy Gradient (DDPG) are utilized for HEV control in [25].…”

Section: Introductionmentioning

confidence: 99%

“…However, the optimality the of RL-based strategy compared to SDP-based strategies is not presented clearly in the literature. Most papers, including [11], [13], [16], [21], and [24], compare the RL-based strategy only with the rule-based strategy. In [13], [18], [20], and [25], the reward function for RL includes the weighted sum of the fuel and electric energy use, thus an equivalent factor or co-state is needed to achieve optimality.…”

Section: Introductionmentioning

confidence: 99%

Comparative Analysis of Energy Management Strategies for HEV: Dynamic Programming and Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

Energy management strategy is an important factor in determining the fuel economy of hybrid electric vehicles; thus, much research on how to distribute the required power to engines and motors of hybrid vehicles is required. Recently, various studies have been conducted based on reinforcement learning to optimally control the hybrid electric vehicle. In fact, the fundamental control approach of reinforcement learning shares many control frameworks with the control approach by using deterministic dynamic programming or stochastic dynamic programming. In this study, we compare the reinforcement learning based strategy by using these dynamic programming-based control approaches. For optimal control of hybrid electric vehicle, each control method was compared in terms of fuel efficiency by performing simulation by using various driving cycles. Based on our simulations, we showed the reinforcement learning-based strategy can obtain global optimality in the optimal control problem with an infinite horizon, which can also be obtained by stochastic dynamic programming. We also showed that the reinforcement learning-based strategy can present a solution close to the optimal one using deterministic dynamic programming, while a reinforcement learning-based strategy is more appropriate for a time variant controller with boundary value constraints. In addition, we verified the convergence characteristics of the control strategy based on reinforcement learning, when transfer learning was performed through value initialization using stochastic dynamic programming. INDEX TERMS Dynamic programming, hybrid electric vehicle, optimal control, reinforcement learning, power management.

show abstract

A deep reinforcement learning framework for optimizing fuel economy of hybrid electric vehicles

Cited by 33 publications

References 21 publications

Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects

Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects

Towards a Smarter Energy Management System for Hybrid Vehicles: A Comprehensive Review of Control Strategies

Comparative Analysis of Energy Management Strategies for HEV: Dynamic Programming and Reinforcement Learning

Contact Info

Product

Resources

About