Batch reinforcement learning for network-safe demand response in unknown electric grids

Lesage‐Landry, Antoine; Callaway, Duncan S.

doi:10.1016/j.epsr.2022.108375

Cited by 9 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collectively, these techniques are categorized into four primary groups, as illustrated in Figure 1: artificial intelligence techniques, conventional methods, metaheuristic-based methods, and others [17]. Within the realm of artificial intelligence, notable algorithms include fuzzy logic [18], game theory [19], and various forms of reinforcement learning, such as Q-Learning [20], DQN [21], actor-critic methods [22], and TD3 [23]. These have been applied with considerable success to the challenge of energy management in microgrids.…”

Section: Literature Reviewmentioning

confidence: 99%

Using Stochastic Dual Dynamic Programming to Solve the Multi-Stage Energy Management Problem in Microgrids

Tabares,

Cortés

2024

Energies

View full text Add to dashboard Cite

In recent years, the adoption of renewable energy sources has significantly increased due to their numerous advantages, which include environmental sustainability and economic viability. However, the management of electric microgrids presents complex challenges, particularly in the orchestration of energy production and consumption under the uncertainty of fluctuating meteorological conditions. This study aims to enhance decision-making processes within energy management systems specifically designed for microgrids that are interconnected with primary grids, addressing the stochastic and dynamic nature of energy generation and consumption patterns among microgrid users. The research incorporates stochastic models for energy pricing in transactions with the main grid and probabilistic representations of energy generation and demand. This comprehensive methodology allows for an accurate depiction of the volatile dynamics prevalent in the energy markets, which are critical in influencing microgrid operational performance. The application of the Stochastic Dual Dynamic Programming (SDDP) algorithm within a multi-stage adaptive framework for microgrids is evaluated for its effectiveness compared to deterministic approaches. The SDDP algorithm is utilized to develop robust strategies for managing the energy requirements of 1, 2, and 12 prosumers over a 24 h planning horizon. A comparative analysis against the precise solutions obtained from dynamic programming via Monte Carlo simulations indicates a strong congruence between the strategies proposed by the SDDP algorithm and the optimal solutions. The results provide significant insights into the optimization of energy management systems in microgrid settings, emphasizing improvements in operational performance and cost reduction.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Using Stochastic Dual Dynamic Programming to Solve the Multi-Stage Energy Management Problem in Microgrids

Tabares,

Cortés

2024

Energies

View full text Add to dashboard Cite

show abstract

“…However, such approaches may require ad hoc tuning of the constraint violation reward and may result in unsafe decisions during the exploration phase. In the second category, the safety of the decisions is promoted by offline (batch) learning to initialize the exploration [16] or by the transfer of expert knowledge learned offline to guide the exploration [17]- [19]. Despite significant improvements, these approaches cannot provide safety guarantees and are not suitable for fully online learning.…”

Section: Introductionmentioning

confidence: 99%

Safe Reinforcement Learning for Strategic Bidding of Virtual Power Plants in Day-Ahead Markets

Stanojev,

Mitridati,

Prata

et al. 2023

2023 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm)

View full text Add to dashboard Cite

“…However, such approaches may require adhoc tuning of the constraint violation reward and may result in unsafe decisions during the exploration phase. In the second category, safety of the decisions is promoted by offline (batch) learning to initialize the exploration [12], or by the transfer of expert knowledge learned offline to guide the exploration [13]- [15]. Despite significant improvements, these approaches cannot provide theoretical safety guarantees, and are not suitable for fully online learning.…”

Section: Introductionmentioning

confidence: 99%

A Reinforcement Learning Approach for Fast Frequency Control in Low-Inertia Power Systems

Stanojev

Ognjen

Markovic

et al. 2021

2020 52nd North American Power Symposium (NAPS)

View full text Add to dashboard Cite

The electric grid is undergoing a major transition from fossil fuel-based power generation to renewable energy sources, typically interfaced to the grid via power electronics. The future power systems are thus expected to face increased control complexity and challenges pertaining to frequency stability due to lower levels of inertia and damping. As a result, the frequency control and development of novel ancillary services is becoming imperative. This paper proposes a data-driven control scheme, based on Reinforcement Learning (RL), for grid-forming Voltage Source Converters (VSCs), with the goal of exploiting their fast response capabilities to provide fast frequency control to the system. A centralized RL-based controller collects generator frequencies and adjusts the VSC power output, in response to a disturbance, to prevent frequency threshold violations. The proposed control scheme is analyzed and its performance evaluated through detailed time-domain simulations of the IEEE 14-bus test system.

show abstract

Batch reinforcement learning for network-safe demand response in unknown electric grids

Cited by 9 publications

References 22 publications

Using Stochastic Dual Dynamic Programming to Solve the Multi-Stage Energy Management Problem in Microgrids

Using Stochastic Dual Dynamic Programming to Solve the Multi-Stage Energy Management Problem in Microgrids

Safe Reinforcement Learning for Strategic Bidding of Virtual Power Plants in Day-Ahead Markets

A Reinforcement Learning Approach for Fast Frequency Control in Low-Inertia Power Systems

Contact Info

Product

Resources

About