Deep Learning for Load Forecasting: Sequence to Sequence Recurrent Neural Networks With Attention

Abstract

The biggest contributor to global warming is energy production and use. Moreover, a push for electrical vehicle and other economic developments are expected to further increase energy use. To combat these challenges, electrical load forecasting is essential as it supports energy production planning and scheduling, assists with budgeting, and helps identify saving opportunities. Machine learning approaches commonly used for energy forecasting such as feedforward neural networks and support vector regression encounter challenges with capturing time dependencies. Consequently, this paper proposes Sequence to Sequence Recurrent Neural Network (S2S RNN) with Attention for electrical load forecasting. The S2S architecture from language translation is adapted for load forecasting and a corresponding sample generation approach is designed. RNN enables capturing time dependencies present in the load data and S2S model further improves time modeling by combining two RNNs: encoder and decoder. The attention mechanism alleviates the burden of connecting encoder and decoder. The experiments evaluated attention mechanisms with different RNN cells (vanilla, LSTM, and GRU) and with varied time horizons. Results show that S2S with Bahdanau attention outperforms other models. Accuracy decreases as forecasting horizon increases; however, longer input sequences do not always increase accuracy. INDEX TERMS Attention mechanism, gated recurrent units, GRU, load forecasting, long short-term memory, LSTM, recurrent neural networks, sequence-to-sequence networks. LJUBISA SEHOVAC (Student Member, IEEE) received the B.Sc. degree in applied math and M.E.Sc. degree in software engineering, collaborative specialization in artificial intelligence

A Comprehensive Review of the Load Forecasting Techniques Using Single and Hybrid Predictive Models

“…An improved DBN especially for Demand Side Management was designed by Kong [32] for STLF, outperforming autoregressive integrated moving average (ARIMA), Least Square SVM and conventional DBM with MAPE and RMSE of 3.864 and 341.601 respectively. RNN and its different approaches are widely used for short-term residential load forecasting [33]. LSTM was further integrated with Gated Recurrent Unit (GRU) for hybrid distribution feeder LTLF by Dong and Grumbach [34].…”

Short Term Load Forecasting Based on SBiGRU and CEEMDAN-SBiGRU Combined Model

“…In formula (13), N is the number of final modal components, and r(t) is the final monotone residual signal.…”

“…The advantage of the recurrent connection is that it can learn and record the historical information in the sequence, so that it can be very good in the prediction of data Characteristics of fitting data on time series [12]. Based on the time series characteristics of load series, RNN has been introduced into load forecasting by many scholars [13]- [17]. In literature [18], firstly, K-means was used to classify the data sets into the same cluster, and then the EEMD algorithm was used to decompose the load series into relatively stable components, and the BiRNN model was established for each component.…”

HSIC Bottleneck Based Distributed Deep Learning Model for Load Forecasting in Smart Grid With a Comprehensive Survey

“…Big data takes a lot of time. ANN is not available at DML based big data load forecasting model [60] because the applied tools and libraries are not supported ANN directly.…”