Deep Reinforcement Learning for Hybrid Energy Storage Systems: Balancing Lead and Hydrogen Storage

Desportes, Louis; Fijalkow, Inbar; Aupert, Pierre

doi:10.3390/en14154706

Cited by 16 publications

(11 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Application Objective Building Type Algorithm [141], [142] Other/Mixed Cost Residential DQN [143] Cost & Load Balance [94] EV, ES, and RG Cost [144] Other [145] Cost & Comfort [146] HVAC, Fans, WH Cost [147] Other/Mixed Commercial [148] Cost & Comfort [149], [150] HVAC, Fans, WH Mixed/NA [151], [152] Other/Mixed Cost [153], [154] P2P Trading Other Mixed/NA [163] EV, ES, and RG [164] Other/Mixed Cost [165] Cost & Comfort Residential TRPO [51], [168], [169], [170] Other/Mixed [171], [172] Cost & Load Balance [173] Cost [174] EV, ES, and RG [175] Other/Mixed Cost & Comfort Academic [176] Other [177], [178] EV, ES, and RG Commercial [179], [180], [181] HVAC, Fans, WH Cost & Comfort Mixed/NA [182], [183], [184] EV, ES, and RG Other [185], [186] Other/Mixed Cost & Load Balance Residential SAC [187], [188] HVAC, Fans, WH Cost Commercial [189], [190],…”

Section: Referencementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The twin capabilities of learning from experience and learning at higher levels of abstraction, set reinforcement learning apart from other areas of machine learning and (within the broader context) all of artificial intelligence. It allows algorithmic agents to replace human beings in the real world, including in homes and buildings, in application domains that had hitherto been considered to be beyond today’s capabilities. This goal, specifically aimed at home energy automation that forms the backdrop of this article, which surveys the use of deep reinforcement learning in various HEMS applications. The article provides an overview of generic reinforcement learning. This is followed with discussions on the state-of-the-art methods for value based, policy gradient, and actor-critic methods in deep reinforcement learning. In order to make published literature in reinforcement learning more accessible to HEMS researchers, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using the same terminology as the machine learning community. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. The survey suggests that this research is still in its infancy.

show abstract

Section: Referencementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Building Type Algorithm [152,153] Other/Mixed Cost Residential DQN [154] Cost and Load Balance [105] EV, ES, and RG Cost [155] Other [156] Cost and Comfort [157] HVAC, Fans, WH Cost [158] Other/Mixed Commercial [159] Cost and Comfort [160,161] EV, ES, and RG [175] Other/Mixed Cost [176] Cost and Comfort Residential TRPO Other/Mixed [182,183] Cost and Load Balance [184] Cost [185] EV, ES, and RG [186] Other/Mixed Cost and Comfort Academic [187] Other [188,189] EV, ES, and RG Commercial [190][191][192] HVAC, Fans, WH Cost and Comfort Mixed/NA [193][194][195] EV, ES, and RG Other [196,197] Other/Mixed Cost and Load Balance Residential SAC [198,199] HVAC, Fans, WH Cost Commercial [103,[200][201][202] Cost and Comfort [203] Other/Mixed [204] Academic [205][206][207] HVAC, Fans, WH Cost and Load Balance Mixed/NA [208][209][210]<...…”

Section: Reference Application Objectivementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The steep rise in reinforcement learning (RL) in various applications in energy as well as the penetration of home automation in recent years are the motivation for this article. It surveys the use of RL in various home energy management system (HEMS) applications. There is a focus on deep neural network (DNN) models in RL. The article provides an overview of reinforcement learning. This is followed with discussions on state-of-the-art methods for value, policy, and actor–critic methods in deep reinforcement learning (DRL). In order to make the published literature in reinforcement learning more accessible to the HEMS community, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using standard machine learning terminology. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. It suggests that research in this direction is still in its infancy. Lastly, the article proposes four performance metrics to evaluate RL methods.

show abstract

“…Simulation has verified the system's success in maintaining the mismatch between generation and demand, which gives a reduction in plant scheduling of 12%, and a rise of Microgrid renewable generation utility of 22.3%. A new deep RL control approach was recently suggested by L. Desportes et al [118] for a power distribution network consisting of a hybrid ESS of lead battery and hydrogen storage, a photovoltaic system as a renewable resource, and a consumer, represented by a partial islanded building. The main aim of the designed approach was to accomplish a 35% long-term renewable feeding for the building and reduce emission impacts due to fuel generation.…”

Section: Deep Q-learningmentioning

confidence: 99%

“…A new deep RL control approach was recently suggested by L. Desportes et al [118] for a power distribution network consisting of a hybrid ESS of lead battery and hydrogen storage, a photovoltaic system as a renewable resource, and a consumer, represented by a partial islanded building. The main aim of the designed approach was to accomplish a 35% long-term renewable feeding for the building and reduce emission impacts due to fuel generation.…”

Section: Actor-critic Algorithmsmentioning

confidence: 99%

Strategies for Controlling Microgrid Networks with Energy Storage Systems: A Review

2021

View full text Add to dashboard Cite

Distributed Energy Storage Systems are considered key enablers in the transition from the traditional centralized power system to a smarter, autonomous, and decentralized system operating mostly on renewable energy. The control of distributed energy storage involves the coordinated management of many smaller energy storages, typically embedded within microgrids. As such, there has been much recent interest related to controlling aspects of supporting power-sharing balance and sustainability, increasing system resilience and reliability, and balancing distributed state of charge. This paper presents a comprehensive review of decentralized, centralized, multiagent, and intelligent control strategies that have been proposed to control and manage distributed energy storage. It also highlights the potential range of services that can be provided by these storages, their control complications, and proposed solutions. Specific focus on control strategies based upon multiagent communication and reinforcement learning is a main objective of this paper, reflecting recent advancements in digitalization and AI. The paper concludes with a summary of emerging areas and presents a summary of promising future directions.

show abstract

Deep Reinforcement Learning for Hybrid Energy Storage Systems: Balancing Lead and Hydrogen Storage

Cited by 16 publications

References 14 publications

Reinforcement Learning: Theory and Applications in HEMS

Reinforcement Learning: Theory and Applications in HEMS

Reinforcement Learning: Theory and Applications in HEMS

Strategies for Controlling Microgrid Networks with Energy Storage Systems: A Review

Contact Info

Product

Resources

About