Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading

Sanayha, Manassakan; Vateekul, Peerapon

doi:10.1109/access.2022.3224460

Cited by 7 publications

(4 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where, Pr n t represents the profit of the n the prosumer. E n selling,t and γ selling t are sold energy to the grid in kWh and energy selling price with TOU pricing scheme [85], respectively. The primary objectives of the electric grid are to supply energy to the prosumers during insufficient energy generation in DERs and insufficient energy transactions in P2P energy trading.…”

Section: ) Electric Grid (Utility)mentioning

confidence: 99%

Optimization of a Cluster-Based Energy Management System Using Deep Reinforcement Learning Without Affecting Prosumer Comfort: V2X Technologies and Peer-to-Peer Energy Trading

Yavuz,

Kivanç

2024

IEEE Access

View full text Add to dashboard Cite

The concept of Prosumer has enabled consumers to actively participate in Peer-to-Peer (P2P) energy trading, particularly as Renewable Energy Source (RES)s and Electric Vehicle (EV)s have become more accessible and cost-effective. In addition to the P2P energy trading, prosumers benefit from the relatively high energy capacity of EVs through the integration of Vehicle-to-X (V2X) technologies, such as Vehicle-to-Home (V2H), Vehicle-to-Load (V2L), and Vehicle-to-Grid (V2G). Optimization of an Energy Management System (EMS) is required to allocate the required energy efficiently within the cluster, due to the complex pricing and energy exchange mechanism of P2P energy trading and multiple EVs with V2X technologies. In this paper, Deep Reinforcement Learning (DRL) based EMS optimization method is proposed to optimize the pricing and energy exchanging mechanisms of the P2P energy trading without affecting the comfort of prosumers. The proposed EMS is applied to a small-scale cluster-based environment, including multiple (6) prosumers, P2P energy trading with novel hybrid pricing and energy exchanging mechanisms, and V2X technologies (V2H, V2L, and V2G) to reduce the overall energy costs and increase the Self-Sufficiency Ratio (SSR)s. Multi Double Deep Q-Network (DDQN) agents based DRL algorithm is implemented and the environment is formulated as a Markov Decision Process (MDP) to optimize the decision-making process. Numerical results show that the proposed EMS reduces the overall energy costs by 19.18%, increases the SSRs by 9.39%, and achieves an overall 65.87% SSR. Additionally, numerical results indicates that model-free DRL, such as DDQN agent based Deep Q-Network (DQN) Reinforcement Learning (RL) algorithm, promise to eliminate the energy management complexities with multiple uncertainties.INDEX TERMS Energy management system, peer-to-peer energy trading, vehicle-to-home, multi-agent reinforcement learning, deep reinforcement learning, smart grids.

show abstract

Section: ) Electric Grid (Utility)mentioning

confidence: 99%

Optimization of a Cluster-Based Energy Management System Using Deep Reinforcement Learning Without Affecting Prosumer Comfort: V2X Technologies and Peer-to-Peer Energy Trading

Yavuz,

Kivanç

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…State in the context of DL in this study is the condition of the fish pond which consists of temperature conditions, pH conditions, and turbidity conditions. For example, the state temperature is 25 • C, the turbidity is 24 NTU, and the pH is 7, which are denoted as S: [7,24,25]. Then, with these conditions, the action is in the form of a declaration of the condition of the fish pool in a "normal" state, which is denoted as A: normal, which then triggers the system to give a reward (R) to become the next condition R t+1 and S t+1 .…”

Section: Deep Reinforcement Learning (Drl)mentioning

confidence: 99%

“…The structure algorithm uses a deep Q network (DQN), and the final results show that the agent can solve problems by controlling the load frequency and determining the appropriate power and frequency requirements. In contrast, Sanayha and Vateekul [25] developed a model-based agent on DRL for peer-to-peer energy trading cases. Agent performance is based on an environment that has been modeled with multivariate-long short-term memory (multivariate-LSTM) with time-series data.…”

Section: Introductionmentioning

confidence: 99%

Designing Aquaculture Monitoring System Based on Data Fusion through Deep Reinforcement Learning (DRL)

2023

View full text Add to dashboard Cite

The aquaculture production sector is one of the suppliers of global food consumption needs. Countries that have a large amount of water contribute to the needs of aquaculture production, especially the freshwater fisheries sector. Indonesia is a country that has a large number of large bodies of water and is the top-five producer of aquaculture production. Technology and engineering continue to be developed to improve the quality and quantity of aquaculture production. One aspect that can be observed is how the condition of fish pond water is healthy and supports fish growth. Various studies have been conducted related to the aquaculture monitoring system, but the problem is how effective it is in terms of accuracy of the resulting output, implementation, and costs. In this research, data fusion (DF) and deep reinforcement learning (DRL) were implemented in an aquaculture monitoring system with temperature, turbidity, and pH parameters to produce valid and accurate output. The stage begins with testing sensor accuracy as part of sensor quality validation, then integrating sensors with wireless sensor networks (WSNs) so they can be accessed in real time. The implemented DF is divided into three layers: first, the signal layer consists of WSNs and their components. Second, the feature layer consists of DRL combined with deep learning (DL). Third, the decision layer determines the output of the condition of the fish pond in “normal” or “not normal” conditions. The analysis and testing of this system look at several factors, i.e., (1) the accuracy of the performance of the sensors used; (2) the performance of the models implemented; (3) the comparison of DF-DRL-based systems with rule-based algorithm systems; and (4) the cost effectiveness compared to labor costs. Of these four factors, the DF-DRL-based aquaculture monitoring system has a higher percentage value and is a low-cost alternative for an accurate aquaculture monitoring system.

show abstract

“…Despite the recent increase in solar energy use in both the commercial and residential sectors, the intermittent behavior of solar energy's radiation intensity continues to pose a significant challenge to power grid operations. Most renewable energy sources suffer from unpredictability or an intermittent nature [10][11][12]. Due to the interplay between radiation and matter, solar irradiance is the radiative energy from the Sun that eventually reaches the photovoltaic (PV) cells [13,14].…”

Section: Introductionmentioning

confidence: 99%

Predicting Solar Irradiance at Several Time Horizons Using Machine Learning Algorithms

2023

View full text Add to dashboard Cite

Photovoltaic (PV) panels need to be exposed to sufficient solar radiation to produce the desired amount of electrical power. However, due to the stochastic nature of solar irradiance, smooth solar energy harvesting for power generation is challenging. Most of the available literature uses machine learning models trained with data gathered over a single time horizon from a location to forecast solar radiation. This study uses eight machine learning models trained with data gathered at various time horizons over two years in Limpopo, South Africa, to forecast solar irradiance. The goal was to study how the time intervals for forecasting the patterns of solar radiation affect the performance of the models in addition to determining their accuracy. The results of the experiments generally demonstrate that the models’ accuracy decreases as the prediction horizons get longer. Predictions were made at 5, 10, 15, 30, and 60 min intervals. In general, the deep learning models outperformed the conventional machine learning models. The Convolutional Long Short-Term Memory (ConvLSTM) model achieved the best Root Mean Square Error (RMSE) of 7.43 at a 5 min interval. The Multilayer Perceptron (MLP) model, however, outperformed other models in most of the prediction intervals.

show abstract

Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading

Cited by 7 publications

References 68 publications

Optimization of a Cluster-Based Energy Management System Using Deep Reinforcement Learning Without Affecting Prosumer Comfort: V2X Technologies and Peer-to-Peer Energy Trading

Optimization of a Cluster-Based Energy Management System Using Deep Reinforcement Learning Without Affecting Prosumer Comfort: V2X Technologies and Peer-to-Peer Energy Trading

Designing Aquaculture Monitoring System Based on Data Fusion through Deep Reinforcement Learning (DRL)

Predicting Solar Irradiance at Several Time Horizons Using Machine Learning Algorithms

Contact Info

Product

Resources

About