Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects

Hu, Xiasong; Li, Teng; Qi, Xuewei; Barth, Matthew

doi:10.1109/mie.2019.2913015

Cited by 191 publications

(70 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, the objectives (O1 and O2) of the study are to combat the drawbacks of conventional practices, i.e., calibration time and the inability to customize the control strategy to a specific driver. Furthermore, as suggested in literature [37], RL methods can improve fuel economy when compared to the rule based methods. However, these RL techniques come at the cost of learning time and this forms the third objective (O3), wherein the learning time must be minimized:…”

Section: Objectivesmentioning

confidence: 94%

“…For continuous-spaces, the actor-critic method was used for the power management in a PHEV [36]. A qualitative study on RL techniques on HEVs and PHEVs shows potential for RL controllers to replace rule based controllers [37]. Similarly, learning based techniques have been used to train neural networks to predict the driving environment and generate an optimal torque split, achieving fuel savings [33].…”

Section: Torque Split Controlmentioning

confidence: 99%

See 1 more Smart Citation

Powertrain Control for Hybrid-Electric Vehicles Using Supervised Machine Learning

Harold¹,

Prakash²,

Hofman³

2020

Vehicles

View full text Add to dashboard Cite

This paper presents a novel framework to enable automatic re-training of the supervisory powertrain control strategy for hybrid electric vehicles using supervised machine learning. The aim of re-training is to customize the control strategy to a user-specific driving behavior without human intervention. The framework is designed to update the control strategy at the end of a driving task. A combination of dynamic programming and supervised machine learning is used to train the control strategy. The trained control strategy denoted as SML is compared to an online-implementable strategy based on the combination of the optimal operation line and Pontryagin’s minimum principle denoted as OOL-PMP, on the basis of fuel consumption. SML consistently performed better than OOL-PMP, evaluated over five standard drive cycles. The EUDC performance was almost identical while on FTP75 the OOL-PMP consumed 14.7% more fuel than SML. Moreover, the deviation from the global benchmark obtained from dynamic programming was between 1.8% and 5.4% for SML and between 5.8% and 16.8% for OOL-PMP. Furthermore, a test-case was conducted to emulate a real-world driving scenario wherein a trained controller is exposed to a new drive cycle. It is found that the performance on the new drive cycle deviates significantly from the optimal policy; however, this performance gap is bridged with a single re-training episode for the respective test-case.

show abstract

Section: Objectivesmentioning

confidence: 94%

Section: Torque Split Controlmentioning

confidence: 99%

Powertrain Control for Hybrid-Electric Vehicles Using Supervised Machine Learning

Harold¹,

Prakash²,

Hofman³

2020

Vehicles

View full text Add to dashboard Cite

show abstract

“…The goal of the agent is to search an optimal sequence of control actions based on feedback from the environment. Owing to its characteristics of self-evaluation and self-promotion, RL is widely used in many research fields [28][29][30][31][32].…”

Section: A Rl Conceptmentioning

confidence: 99%

Decision-Making Strategy on Highway for Autonomous Vehicles Using Deep Reinforcement Learning

Liao

Tang

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Autonomous driving is a promising technology to reduce traffic accidents and improve driving efficiency. In this work, a deep reinforcement learning (DRL)-enabled decision-making policy is constructed for autonomous vehicles to address the overtaking behaviors on the highway. First, a highway driving environment is founded, wherein the ego vehicle aims to pass through the surrounding vehicles with an efficient and safe maneuver. A hierarchical control framework is presented to control these vehicles, which indicates the upper-level manages the driving decisions, and the lower-level cares about the supervision of vehicle speed and acceleration. Then, the particular DRL method named dueling deep Qnetwork (DDQN) algorithm is applied to derive the highway decision-making strategy. The exhaustive calculative procedures of deep Q-network and DDQN algorithms are discussed and compared. Finally, a series of estimation simulation experiments are conducted to evaluate the effectiveness of the proposed highway decision-making policy. The advantages of the proposed framework in convergence rate and control performance are illuminated. Simulation results reveal that the DDQN-based overtaking policy could accomplish highway driving tasks efficiently and safely. INDEX TERMS Autonomous driving, decision-making, deep reinforcement learning, dueling deep Qnetwork, deep Q-learning, overtaking policy.

show abstract

“…Rule-based strategies rely on predefined rules which require abundant human experiences and high calibration efforts and not adaptive to varied driving conditions and vehicle types [7][8][9]. Optimization-based strategies are considered as practical alternatives which can obtain the optimal control through the known or predicted driving conditions [10].…”

Section: Introductionmentioning

confidence: 99%

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Guo

Tang

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL). This work aims to address the defect of DRL in tedious training time. First, an optimization control modeling of a hybrid tracked vehicle is built, wherein the elaborate powertrain components are introduced. Then, a bi-level control framework is constructed to derive the energy management strategies (EMSs). The upper-level is applying the particular deep deterministic policy gradient (DDPG) algorithms for EMS training at different speed intervals. The lower-level is employing the TL method to transform the pre-trained neural networks for a novel driving cycle. Finally, a series of experiments are executed to prove the effectiveness of the presented control framework. The optimality and adaptability of the formulated EMS are illuminated. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.

show abstract

Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects

Cited by 191 publications

References 48 publications

Powertrain Control for Hybrid-Electric Vehicles Using Supervised Machine Learning

Powertrain Control for Hybrid-Electric Vehicles Using Supervised Machine Learning

Decision-Making Strategy on Highway for Autonomous Vehicles Using Deep Reinforcement Learning

Transfer Deep Reinforcement Learning-Enabled Energy Management Strategy for Hybrid Tracked Vehicle

Contact Info

Product

Resources

About