This paper proposes an energy management strategy for a power-split plug-in hybrid electric vehicle (PHEV) based on reinforcement learning (RL). Firstly, a control-oriented power-split PHEV model is built, and then the RL method is employed based on the Markov Decision Process (MDP) to find the optimal solution according to the built model. During the strategy search, several different standard driving schedules are chosen, and the transfer probability of the power demand is derived based on the Markov chain. Accordingly, the optimal control strategy is found by the Q-learning (QL) algorithm, which can decide suitable energy allocation between the gasoline engine and the battery pack. Simulation results indicate that the RL-based control strategy could not only lessen fuel consumption under different driving cycles, but also limit the maximum discharge power of battery, compared with the charging depletion/charging sustaining (CD/CS) method and the equivalent consumption minimization strategy (ECMS).

In this paper, a stochastic model predictive control (MPC) method based on reinforcement learning is proposed for energy management of plug-in hybrid electric vehicles (PHEVs). Firstly, the power transfer of each component in a power-split PHEV is described in detail. Then an effective and convergent reinforcement learning controller is trained by the Q-learning algorithm according to the driving power distribution under multiple driving cycles. By constructing a multi-step Markov velocity prediction model, the reinforcement learning controller is embedded into the stochastic MPC controller to determine the optimal battery power in predicted time domain. Numerical simulation results verify that the proposed method achieves superior fuel economy that is close to that by stochastic dynamic programming method. In addition, the effective state of charge tracking in terms of different reference trajectories highlight that the proposed method is effective for online application requiring a fast calculation speed.

The implementation of an energy management strategy plays a key role in improving the fuel economy of plug-in hybrid electric vehicles (PHEVs). In this article, a bi-level energy management strategy with a novel speed prediction method leveraged by reinforcement learning is proposed to construct the optimization scheme for the inner energy allocation of PHEVs. First, the powertrain transmission model of the PHEV in a power-split type is analyzed in detail to obtain the energy routing and its crucial characteristics. Second, a Q-learning (QL) algorithm is applied to establish the speed predictor. Third, the double QL algorithm is introduced to train an effective controller offline that realizes the optimal power distribution. Finally, given a reference battery's state of charge (SOC), a model predictive control framework solved by the reinforcement learning agent with a novel speed predictor is proposed to build the bi-level energy management strategy. The simulation results show that the proposed method performs with a satisfying fuel economy in different driving scenarios while tracking the corresponding SOC references. Moreover, the calculation performance also implies the potential online capability of the proposed method.

No abstract