Sensitivity Analysis of Reinforcement Learning-Based Hybrid Electric Vehicle Powertrain Control

A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation

“…Actuator Continuity [17] battery-UC EV P bat [105] FC HEV ∆P bat [110] FC-battery-UC HEV P bat , P FC [111] FC HEV SoC min , SoC max [112] EV τ EM [96] ORC-WHR ṁworking f luid [18,43] parallel HEV i bat , n gear discrete combined Dyna-Q [52,59,60] series HEV P engine discrete continuous [53] power-split PHEV n operating mode Q-learning (approximate) [54] parallel HEV τ EM discrete continuous [113] EV ∆SoC overnight Q-learning vs. DQN [114] EV with two batteries P split discrete continuous DQN [72,91,115,116] parallel HEV τ engine discrete continuous [63,73,87] power-split HEV ∆P engine [117] power-split PHEV τ engine , ω engine [61,100] series HEV pos throttle [102] EV i charge/discharge [118] EV thermal management ω f an , ω compressor [44,45] [47,48,90] parallel HEV P EM , cool battery [79] series HEV ∆P engine [65] power-split HEV P engine [103] ORC-WHR ω pump [83] 8 × 8 EV τ wheel,x [73,97] power-split HEV τ engine , ω engine , τ EM [64] power-split HEV n operating mode , τ engine , ω engine continuous combined TD3 [72,74,…”

“…Despite power-split being a continuous control action, many researchers have successfully trained RL-based EMS using agents with discrete action outputs to determine power-split. While reported performance improvement is highly dependent on the quality of the baseline controller RL is compared to, RL has compared favorably to ECMS [74], MPC [76], SDP [81], and rule-based EMS [78] despite the optimality loss associated with a discrete-to-continuous continuity conversion. However, as RL-based powertrain control matures it will be tasked with governing systems with greater control complexity, increasing the risk that the curse of dimensionality will prohibit the use of RL algorithms limited to discrete action outputs.…”

“…f uel,electrical − w 1 |δSOC| parallel HEV[74,75] − ṁ f uel − ṁe. f uel,electrical + w 1 δSoC 2 − parallel HEV [76,77] 1 − (w 1 ṁ f uel + w 2 ṁe.…”