Real-time energy purchase optimization for a storage-integrated photovoltaic system by deep reinforcement learning

Kołodziejczyk, Waldemar; Żółtowska, Izabela; Cichosz, Paweł

doi:10.1016/j.conengprac.2020.104598

Cited by 31 publications

(12 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL is an excellent method for dealing with stochastic control problems. When its "trial‐and‐error and learning‐as‐you‐do" method is applicable, it benefits from requiring little or no domain knowledge [43].…”

Section: Proposed Deep Q‐network Methodologymentioning

confidence: 99%

See 1 more Smart Citation

Deep Q‐network application for optimal energy management in a grid‐tied solar PV‐Battery microgrid

Muriithi

Chowdhury

2022

The Journal of Engineering

View full text Add to dashboard Cite

This paper presents a deep Q‐network (DQN) technique to optimally manage energy resources in a microgrid in which the algorithm learns tasks in the same way as humans do. Every move the agent makes in the environment generates feedback which then motivate the agent to learn more about the environment and perform far more intelligent steps later in its learning stages. This paper proposes a DQN‐based energy management system that learns system uncertainties, including load demand, grid prices and volatile renewable power supply to ensure that energy is optimally dispatched in such a setting. The method uses experience replay and target network to increase learning speed and improve stability in previous research. The performance of the system is evaluated with slow, medium, and fast fluctuating load profiles and their combinations and also with different sets of actions. The system is designed to minimize both grid transactional costs and battery degradation costs. Results shows that the proposed system successfully learns from experience to raise the battery state of charge and optimally shift loads from a one‐time instance, reducing aggregate peak load and cost of energy purchased from the utility grid while maximizing revenue from energy exported to the grid.

show abstract

Section: Proposed Deep Q‐network Methodologymentioning

confidence: 99%

“…RL is an excellent method for dealing with stochastic control problems. When its "trial-and-error and learning-as-you-do" method is applicable, it benefits from requiring little or no domain knowledge [43]. Q-learning is a well-known RL technique in which the policy is modified by the value function known as the Q-function.…”

Section: Q-learning Overviewmentioning

confidence: 99%

Deep Q‐network application for optimal energy management in a grid‐tied solar PV‐Battery microgrid

Muriithi

Chowdhury

2022

The Journal of Engineering

View full text Add to dashboard Cite

show abstract

“…It is crucial to properly select the algorithm parameters affecting the training performance [37]. The learning rate is a key parameter for LSTM networks.…”

Section: Methodsmentioning

confidence: 99%

A Fault Diagnosis Design Based on Deep Learning Approach for Electric Vehicle Applications

2021

View full text Add to dashboard Cite

Diagnosing faults in electric vehicles (EVs) is a great challenge. The purpose of this paper is to demonstrate the detection of faults in an electromechanical conversion chain for conventional or autonomous EVs. The information and data coming from different sensors make it possible for EVs to recover a series of information including currents, voltages, speeds, and so on. This information is processed to detect any faults in the electromechanical conversion chain. The novelty of this study is to develop an architecture for a fault diagnosis model by means of the feature extraction technique. In this regard, the long short-term memory (LSTM) approach for the fault diagnosis is proposed. This approach has been tested for an EV prototype in practice, is superior in accuracy over other fault diagnosis techniques, and is based on machine learning. An EV in an urban context is modeled, and then the fault diagnosis approach is applied based on deep learning architectures. The EV and the fault diagnosis model is simulated in Matlab software. It is also revealed how deep learning contributes to the fault diagnosis of EVs. The simulation and practical results confirm that higher accuracy in the fault diagnosis is obtained by applying the LSTM.

show abstract

“…Wu et al (2020) and Seyyedeh Barhagh et al (2020), present similar approaches with the additional capability to consider dynamic electricity prices. Kolodziejczyk et al (2021) operate a system consisting of photovoltaic generation and battery storage on a real-time electricity market with hourly changing prices. An example of a close to real-time electricity market is Singapore's half-hourly spot market (Zhang et al, 2021); at this time-scale, photovoltaic generation nowcasting exploiting sky image analysis begins to be advantageous.…”

Section: Nowcasting For Vpp With Photovoltaic Generationmentioning

confidence: 99%

Solar Irradiance Nowcasting for Virtual Power Plants Using Multimodal Long Short-Term Memory Networks

et al. 2021

View full text Add to dashboard Cite

The rapid penetration of photovoltaic generation reduces power grid inertia and increases the need for intelligent energy resources that can cope in real time with the imbalance between power generation and consumption. Virtual power plants are a technology for coordinating such resources and monetizing them, for example on electricity markets with real-time pricing or on frequency reserves markets. Accurate short-term photovoltaic generation forecasts are essential for such virtual power plants. Although significant research has been done on medium- and long-term photovoltaic generation forecasting, the short-term forecasting problem requires special attention to sudden fluctuations due to the high variability of cloud cover and related weather events. Solar irradiance nowcasting aims to resolve this variability by providing reliable short-term forecasts of the expected power generation capacity. Sky images captured in proximity to the photovoltaic panels are used to determine cloud behavior and solar intensity. This is a computationally challenging task for conventional computer vision techniques and only a handful of Artificial Intelligence (AI) methods have been proposed. In this paper, a novel multimodal approach is proposed based on two Long Short-Term Memory Networks (LSTM) that receives a temporal image modality of a stream of sky images, a temporal numerical modality of a time-series of past solar irradiance readings and cloud cover readings as inputs for irradiance nowcasting. The proposed nowcasting pipeline consists of a preprocessing module and an irradiance augmentation module that implements methods for cloud detection, Sun localization and mask generation. The complete approach was empirically evaluated on a real-world solar irradiance case study across the four seasons of the northern hemisphere, resulting in a mean improvement of 39% for multimodality.

show abstract

Real-time energy purchase optimization for a storage-integrated photovoltaic system by deep reinforcement learning

Cited by 31 publications

References 28 publications

Deep Q‐network application for optimal energy management in a grid‐tied solar PV‐Battery microgrid

Deep Q‐network application for optimal energy management in a grid‐tied solar PV‐Battery microgrid

A Fault Diagnosis Design Based on Deep Learning Approach for Electric Vehicle Applications

Solar Irradiance Nowcasting for Virtual Power Plants Using Multimodal Long Short-Term Memory Networks

Contact Info

Product

Resources

About