Research on Demand Response Strategy of HVAC Based on Deep Reinforcement Learning

The efficiency of grid-connected photovoltaic (PV) systems can be improved using a suitable maximum power point tracking (MPPT) control. Many MPPT techniques have been developed. However, when atmospheric conditions change rapidly, the PV modules respond nonlinearly and the performance of these methods especially the conventional ones is significantly reduced. Therefore, to successfully pursue the maximum power point (MPP) in PV systems, a robust MPPT integrated with sliding mode control (SMC) is employed. This paper proposes a circle search algorithm based super twisting SMC (CSA-STSMC) for tracking the MPP for 100-kW of different commercial grid-connected PV modules coupled with an energy storage system that used to smooth their output fluctuations. Moreover, the STSMC and PI controllers' gains are optimized using the CSA and Grey Wolf Optimizer (GWO) methods and the performance of proposed method is demonstrated through comparisons with CSA based Proportional-Integral controller (CSA-PI), GWO-STSMC, and Incremental Conductance based PI controller (InC-PI) methods. The optimization and simulation analysis are executed in A MATLAB/Simulink environment. It is observed that the proposed method outperforms the considered methods in terms of robustness, tracking speed, best convergence to the minimum error's value and better tracking efficiency in the maximizing the extracted power from the introduced PV systems.INDEX TERMS Circle search algorithm, Grey wolf optimizer, Maximum power point tracking, Photovoltaic system, PI controller, Super twisting sliding mode controller.

Section: Introductionmentioning

confidence: 99%

Circle Search Algorithm-Based Super Twisting Sliding Mode Control for MPPT of Different Commercial PV Modules

Ghazi,

Al-Ammar,

Hasanien

et al. 2024

“…Furthermore, novel DRL and transfer RL-based MPPT methods for tackling the MPP issue of the PV systems under PSCs were introduced in [34] and [35], respectively. Similarly, W. Pan et al [36] presented an RL-based MPPT algorithm using a DQN with continuous state and discrete action spaces. The proposed method showed a significant improvement in tracking accuracy when compared with the P&O and InC methods and it converges to the MPP faster and has better steady-state performance under PSCs when compared to PSO and GWO methods.…”

Section: Introductionmentioning

confidence: 99%

Dandelion Optimizer-Based Reinforcement Learning Techniques for MPPT of Grid- Connected Photovoltaic Systems

Ghazi,

Al-Ammar,

Hasanien

et al. 2024

The integration of photovoltaic (PV) into electric power systems has been widely explored and adopted to address the problems associated with the depletion of fossil fuels and the release of greenhouse gases. PV panels convert sunlight into electricity, minimizing the reliance on fossil fuels and mitigating environmental pollution. It is crucial to optimally utilize the PV power in the system; hence maximum power point tracking (MPPT) algorithms have been developed to ensure optimal performance of grid-connected PV systems at the maximum power point (MPP) despite changes in weather conditions. Moreover, deep reinforcement learning (DRL) developments provide a promising approach for optimizing grid-connected PV systems, replacing the conventional proportional-integral-derivative (PID) controllers. However, there is limited research evaluating the efficiency of these systems using DRL techniques. This paper proposes a new dandelion optimizer (DO)-based DRL for MPPT of grid-connected photovoltaic systems and evaluates the proposed method for a 100-MW PV plant connected to a 33-kV distribution system. The proposed DRL technique uses proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms for continuous states and discrete or continuous action spaces to adjust the PV-measured voltage based on a reference one produced via DO-PPO and DO-DDPG methods. To test the effectiveness and practicality of the introduced methods, simulations were conducted using actual input data of a 100 MW PV plant connected to a 33-kV distribution system for typical days in summer and winter seasons using MATLAB/Simulink software. The proposed implemented methods were evaluated by comparing their simulation results with other techniques: DO-PID, particle swarm optimization (PSO), and incremental conductance (InC-PI). The findings revealed that the efficiencies of the DC-DC boost and the voltage source converters using the introduced methods were 84.25%-85.90%, and 78.33%-81.10% on a summer day while they were 92.77%-95% and 86.70%-89.50% on a winter day, respectively, which proves that these methods were efficient and effective, indicating their promising potential for future applications.

Optimization of a Cluster-Based Energy Management System Using Deep Reinforcement Learning Without Affecting Prosumer Comfort: V2X Technologies and Peer-to-Peer Energy Trading

Yavuz,

Kivanç

2024

The concept of Prosumer has enabled consumers to actively participate in Peer-to-Peer (P2P) energy trading, particularly as Renewable Energy Source (RES)s and Electric Vehicle (EV)s have become more accessible and cost-effective. In addition to the P2P energy trading, prosumers benefit from the relatively high energy capacity of EVs through the integration of Vehicle-to-X (V2X) technologies, such as Vehicle-to-Home (V2H), Vehicle-to-Load (V2L), and Vehicle-to-Grid (V2G). Optimization of an Energy Management System (EMS) is required to allocate the required energy efficiently within the cluster, due to the complex pricing and energy exchange mechanism of P2P energy trading and multiple EVs with V2X technologies. In this paper, Deep Reinforcement Learning (DRL) based EMS optimization method is proposed to optimize the pricing and energy exchanging mechanisms of the P2P energy trading without affecting the comfort of prosumers. The proposed EMS is applied to a small-scale cluster-based environment, including multiple (6) prosumers, P2P energy trading with novel hybrid pricing and energy exchanging mechanisms, and V2X technologies (V2H, V2L, and V2G) to reduce the overall energy costs and increase the Self-Sufficiency Ratio (SSR)s. Multi Double Deep Q-Network (DDQN) agents based DRL algorithm is implemented and the environment is formulated as a Markov Decision Process (MDP) to optimize the decision-making process. Numerical results show that the proposed EMS reduces the overall energy costs by 19.18%, increases the SSRs by 9.39%, and achieves an overall 65.87% SSR. Additionally, numerical results indicates that model-free DRL, such as DDQN agent based Deep Q-Network (DQN) Reinforcement Learning (RL) algorithm, promise to eliminate the energy management complexities with multiple uncertainties.INDEX TERMS Energy management system, peer-to-peer energy trading, vehicle-to-home, multi-agent reinforcement learning, deep reinforcement learning, smart grids.