Multi-Agent Deep Reinforcement Learning for Coordinated Energy Trading and Flexibility Services Provision in Local Electricity Markets

Ye, Yujian; Papadaskalopoulos, Dimitrios; Yuan, Quan; Tang, Yi; Štrbac, Goran

doi:10.1109/tsg.2022.3149266

Cited by 34 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that in the Illinois 200-bus system, there are 38 generators that can be adjusted in autonomous voltage FIGURE 6 The IEEE 14-bus rewards achieved by the deep deterministic policy gradient agent considering loss and corresponding iterations respectively.…”

Section: 2mentioning

confidence: 99%

“…Deep reinforcement learning based applications in power system are spreading widely because of its capability in solving challenges like decision making in complex and dynamic environment. Recent studies have demonstrated the effective use of DRL-based techniques in resolving various power system issues with satisfactory results, including grid operation [1,2], grid emergency control [3][4][5], energy trading [6][7][8], electricity markets [9], battery control [10,11], demand response [12], economic dispatch [13], cyber security [14][15][16], load-frequency control [17], and real-time topology control [18]. Some of the advantages of the DRL method over traditional methods include flexibility, continuous learning, and the elimination of the need for explicit models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep reinforcement learning based voltage control revisited

Nematshahi,

Shi,

Wang

et al. 2023

IET Generation Trans & Dist

View full text Add to dashboard Cite

Deep Reinforcement Learning (DRL) has shown promise for voltage control in power systems due to its speed and model‐free nature. However, learning optimal control policies through trial and error on a real grid is infeasible due to the mission‐critical nature of power systems. Instead, DRL agents are typically trained on a simulator, which may not accurately represent the real grid. This discrepancy can lead to suboptimal control policies and raises concerns for power system operators. In this paper, we revisit the problem of RL‐based voltage control and investigate how model inaccuracies affect the performance of the DRL agent. Extensive numerical experiments are conducted to quantify the impact of model inaccuracies on learning outcomes. Specifically, techniques that enable the DRL agent are focused on learning robust policies that can still perform well in the presence of model errors. Furthermore, the impact of the agent's decisions on the overall system loss are analyzed to provide additional insight into the control problem. This work aims to address the concerns of power system operators and make DRL‐based voltage control more practical and reliable.

show abstract

Section: 2mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep reinforcement learning based voltage control revisited

Nematshahi,

Shi,

Wang

et al. 2023

IET Generation Trans & Dist

View full text Add to dashboard Cite

show abstract

“…Han et al 36 proposed a novel approach to model smart buildings to assess energy consumption based on the concept of physical-data fusion modeling (PFM). Ye et al 37 proposed a theoretical benchmark for optimizing the coordination of local electricity markets (LEM) using a system-centric model. The approach serves as a model-free coordination method for consumer-centric LEM.…”

Section: Related Workmentioning

confidence: 99%

Short term energy consumption forecasting using neural basis expansion analysis for interpretable time series

Shaikh

Nazir

Khan

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Smart grids and smart homes are getting people’s attention in the modern era of smart cities. The advancements of smart technologies and smart grids have created challenges related to energy efficiency and production according to the future demand of clients. Machine learning, specifically neural network-based methods, remained successful in energy consumption prediction, but still, there are gaps due to uncertainty in the data and limitations of the algorithms. Research published in the literature has used small datasets and profiles of primarily single users; therefore, models have difficulties when applied to large datasets with profiles of different customers. Thus, a smart grid environment requires a model that handles consumption data from thousands of customers. The proposed model enhances the newly introduced method of Neural Basis Expansion Analysis for interpretable Time Series (N-BEATS) with a big dataset of energy consumption of 169 customers. Further, to validate the results of the proposed model, a performance comparison has been carried out with the Long Short Term Memory (LSTM), Blocked LSTM, Gated Recurrent Units (GRU), Blocked GRU and Temporal Convolutional Network (TCN). The proposed interpretable model improves the prediction accuracy on the big dataset containing energy consumption profiles of multiple customers. Incorporating covariates into the model improved accuracy by learning past and future energy consumption patterns. Based on a large dataset, the proposed model performed better for daily, weekly, and monthly energy consumption predictions. The forecasting accuracy of the N-BEATS interpretable model for 1-day-ahead energy consumption with “day as covariates” remained better than the 1, 2, 3, and 4-week scenarios.

show abstract

“…However, since the MPEC problems are nonconvex, the algorithm may not necessarily converge to a locally optimal point but to a hurdle point instead. In this case, an equilibrium checking method is proposed based on diagonal optimal strategy verification [134,135] . The MPEC problem of each firm is solved sequentially by holding rival firms' offers as the EPEC solution.…”

Section: Equilibrium Verification and Choice In Epecmentioning

confidence: 99%

The competition and equilibrium in power markets under decarbonization and decentralization

Chen

Fang

Guo

et al. 2022

iEnergy

View full text Add to dashboard Cite

Equilibrium analysis has been widely studied as an effective tool to model gaming interactions and predict market results. However, as competition modes are fundamentally changed by the decarbonization and decentralization of power systems, analysis techniques must evolve. This article comprehensively reviews recent developments in modelling methods, practical settings and solution techniques in equilibrium analysis. Firstly, we review equilibrium in the evolving wholesale power markets which feature new entrants, novel trading products and multi-stage clearing. Secondly, the competition modes in the emerging distribution market and distributed resource aggregation are reviewed, and we compare peer-to-peer clearing, cooperative games and Stackelberg games. Furthermore, we summarize the methods to treat various information acquisition degrees, risk preferences and rationalities of market participants. To deal with increasingly complex market settings, this review also covers refined analytical techniques and agent-based models used to compute the equilibrium. Finally, based on this review, this paper summarizes key issues in the gaming and equilibrium analysis in power markets under decarbonization and decentralization.

show abstract

Multi-Agent Deep Reinforcement Learning for Coordinated Energy Trading and Flexibility Services Provision in Local Electricity Markets

Cited by 34 publications

References 42 publications

Deep reinforcement learning based voltage control revisited

Deep reinforcement learning based voltage control revisited

Short term energy consumption forecasting using neural basis expansion analysis for interpretable time series

The competition and equilibrium in power markets under decarbonization and decentralization

Contact Info

Product

Resources

About