Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

Abstract

The transition to renewable production and smart grids is driving a massive investment to battery storages, and reinforcement learning (RL) has recently emerged as a potentially disruptive technology for their control and optimization of battery storage systems. A surge of papers has appeared in the last two years applying reinforcement learning to the optimization of battery storages in buildings, energy communities, energy harvesting Internet of Things networks, renewable generation, microgrids, electric vehicles and plug-in hybrid electric vehicles. This article reviews these applications through 4 different perspectives. Firstly, the type of optimization problem is analyzed; the literature can be divided to approaches that optimize either financial targets or energy efficiency. Secondly, the approaches for handling user comfort are analyzed for applications that may impact a human user. Thirdly, this paper discusses the approach to model and reduce battery degradation. Fourthly, the articles are categorized by application context and applications likely to attract a high amount of research are identified. The paper concludes with a list of unresolved challenges.

A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation

“…Battery degradation has been difficult to include when optimizing powertrain performance as modeling battery degradation is dependent on many factors, such as charging voltage, battery composition, depth of discharge, and current density. These dynamics can be simplified into empirical fits [133], but RL-based approaches provide the opportunity to consider physics-based battery degradation models [134].…”