Reinforcement Learning-Based Energy Management of Smart Home with Rooftop Solar Photovoltaic System, Energy Storage System, and Home Appliances

Lee, Sangyoon; Choi, Dae-Hyun

doi:10.3390/s19183937

Cited by 82 publications

(52 citation statements)

References 38 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The objective function (1) for the HEMS optimization problem comprises two terms, each of which includes different decision variables (E net t , T in t ) [25]:…”

Section: Objective Functionmentioning

confidence: 99%

“…s A uc ) and the predicted PV generation output E PV t at time t. In Equation 3, the total energy consumption of all appliances in Equation 2is decomposed into four different types of consumptions corresponding to (i) reducible appliances (a ∈ A c r ), (ii) shiftable appliances with a non-interruptible load (a ∈ A c,N I s ), (iii) shiftable appliances with an interruptible load (a ∈ A c,I s ), and (iv) uncontrollable appliances (a ∈ A uc ) [25]:…”

Section: Net Energy Consumption Equation (2) Expresses the Constraintmentioning

confidence: 99%

“…Another RL-based method, referred to as Q-learning, was applied to HEMS problems. It was integrated with the ANN module for estimating the consumer's comfort level, maintaining the energy efficiency of household appliances [24,25] and predicting the pricing in real time [26]. A new demand response strategy for HEMS was proposed through the combination of Q-learning and fuzzy reasoning, which reduces the number of state-action pairs and fuzzy logic for reward functions [27].…”

Section: Introductionmentioning

confidence: 99%

“…However, to the best of the authors' knowledge, no study has presented an DRL-based algorithm that considers the continuous operations of heterogeneous home appliances and DERs according to the consumer's comfort and preferences. In prior studies [24][25][26], the operations of home appliances and DERs were scheduled using a simple Q-learning method based on an unrealistic discrete action space. Furthermore, prior studies [28,29] focused on the scheduling of the energy consumption of buildings without considering the operation characteristics of home appliances in detail.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach

Lee

Choi

2020

Sensors

Self Cite

View full text Add to dashboard Cite

This paper presents a hierarchical deep reinforcement learning (DRL) method for the scheduling of energy consumptions of smart home appliances and distributed energy resources (DERs) including an energy storage system (ESS) and an electric vehicle (EV). Compared to Q-learning algorithms based on a discrete action space, the novelty of the proposed approach is that the energy consumptions of home appliances and DERs are scheduled in a continuous action space using an actor–critic-based DRL method. To this end, a two-level DRL framework is proposed where home appliances are scheduled at the first level according to the consumer’s preferred appliance scheduling and comfort level, while the charging and discharging schedules of ESS and EV are calculated at the second level using the optimal solution from the first level along with the consumer environmental characteristics. A simulation study is performed in a single home with an air conditioner, a washing machine, a rooftop solar photovoltaic system, an ESS, and an EV under a time-of-use pricing. Numerical examples under different weather conditions, weekday/weekend, and driving patterns of the EV confirm the effectiveness of the proposed approach in terms of total cost of electricity, state of energy of the ESS and EV, and consumer preference.

show abstract

“…The objective function (1) for the HEMS optimization problem comprises two terms, each of which includes different decision variables (E net t , T in t ) [25]:…”

Section: Objective Functionmentioning

confidence: 99%

Section: Net Energy Consumption Equation (2) Expresses the Constraintmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach

Lee

Choi

2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…As basic energy collection component of power generation system, the accurate and reliable modeling of PV module is quite significant in design, optimized simulation, operation and evaluation of photovoltaic power generation system [2][3][4]. The accurate results of model output could be used to get system information like maximum power point tracking (MPPT), power prediction [5][6][7][8][9][10][11]. In addition, some researchers suggest that the random forest (RF) ensemble learning algorithm and the emerging kernel based extreme learning machine (KELM) are explored for the detection and diagnosis of PV arrays early faults (including line-line faults, degradation, open circuit and partial shading).…”

Section: Introductionmentioning

confidence: 99%

Prediction of I–V Characteristic Curve for Photovoltaic Modules Based on Convolutional Neural Network

Jia

et al. 2020

Sensors

View full text Add to dashboard Cite

Photovoltaic (PV) modules are exposed to the outside, which is affected by radiation, the temperature of the PV module back-surface, relative humidity, atmospheric pressure and other factors, which makes it difficult to test and analyze the performance of photovoltaic modules. Traditionally, the equivalent circuit method is used to analyze the performance of PV modules, but there are large errors. In this paper-based on machine learning methods and large amounts of photovoltaic test data-convolutional neural network (CNN) and multilayer perceptron (MLP) neural network models are established to predict the I-V curve of photovoltaic modules. Furthermore, the accuracy and the fitting degree of these methods for current-voltage (I-V) curve prediction are compared in detail. The results show that the prediction accuracy of the CNN and MLP neural network model is significantly better than that of the traditional equivalent circuit models. Compared with MLP models, the CNN model has better accuracy and fitting degree. In addition, the error distribution concentration of CNN has better robustness and the pre-test curve is smoother and has better nonlinear segment fitting effects. Thus, the CNN is superior to MLP model and the traditional equivalent circuit model in complex climate conditions. CNN is a high-confidence method to predict the performance of PV modules. from the measured current-voltage (I-V) characteristic curves is important for evaluation, modeling and diagnosis of the actual operating state of in situ PV arrays [14]. Nowadays, manufacturers of photovoltaic modules provide standard reporting condition (SRC) or standard test condition (STC) ratings of PV modules. The main test conditions of PV modules are mainly under laboratory environment, which include irradiation intensity of 1000 W/m 2 , spectral distribution in accordance with the AM1.5 spectrum and temperature of 25 ± 1 • C for the PV module. However, in the practical engineering environment (outdoor weather conditions), these conditions rarely appear at the same time. The adequacy and applicability of PV modules under STC is still a controversial issue [15]. Actually, the PV modules based on SRC are still unreasonable for the real-world weather conditions. Therefore, performance test of PV module based on STC condition needs to be improved in the outdoor weather conditions [16]. Although the outdoor characteristics of PV modules can be predicted by algebraic or numerical methods, these methods used in photovoltaic system engineering ignore some second-order effects, like wind speed, shunt resistance, parasitic capacitance, spectral effect and non-linearity under low illumination, thus the prediction error of these methods is large [17]. Based on this, many photovoltaic modeling methods have been proposed in recent years. Generally, these methods can be divided into two types: the white box model based on equivalent circuit and the black box model based on regression [18][19][20]. The equivalent circuit method is used for the prediction of photovolta...

show abstract

Reinforcement Learning for Intelligent Building Energy Management System Control *

Kotevska

Andelfinger

2022

Intelligent Data Mining and Analysis in Power and Energy Systems

View full text Add to dashboard Cite

Reinforcement Learning-Based Energy Management of Smart Home with Rooftop Solar Photovoltaic System, Energy Storage System, and Home Appliances

Cited by 82 publications

References 38 publications

Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach

Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach

Prediction of I–V Characteristic Curve for Photovoltaic Modules Based on Convolutional Neural Network

Reinforcement Learning for Intelligent Building Energy Management System Control *

Contact Info

Product

Resources

About