Improving time series forecasting using LSTM and attention models

Abbasimehr, Hossein; Paki, Reza

doi:10.1007/s12652-020-02761-x

Cited by 99 publications

(41 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Attention mechanisms are an input processing approach for neural networks that focus on a particular categorized attribute of the entire dataset [20], [21]. It aims to break down a large task into smaller ones [22].…”

Section: A Attention-based Lstmmentioning

confidence: 99%

Robust LSTM With Tuned-PSO and Bifold-Attention Mechanism for Analyzing Multivariate Time-Series

et al. 2022

View full text Add to dashboard Cite

The need for accurate time-series results is badly demanding. LSTM has been applied for forecasting time series, which is generated when variables are observed at discrete and equal time intervals. Nevertheless, the problem of determining hyperparameters with a relatively high random rate will reduce the accuracy of the prediction results. This paper aims to promote LSTM with tuned-PSO and Bifold-Attention mechanism. PSO optimizes LSTM hyperparameters, and Bifold-attention mechanism selects the optimal input for LSTM. An accurate, adaptive, and robust time-series forecasting model is the main contribution, compared with ARIMA, MLP, LSTM, PSO-LSTM, A-LSTM, and PSO-A-LSTM. The model comparison is based on the accuracy of each model in forecasting Beijing PM2.5, Beijing Multi-Site, Air Quality, Appliances Energy, Wind Speed, and Traffic Flow. Proposed PSO Bifold-Attention LSTM has lower MAPE and RMSE than baselines. In other words, the model outperformed all LSTM base models in this study. The proposed model's accuracy is adaptable in daily, weekly, and monthly multivariate time-series datasets. This ground-breaking innovation is valuable for time-series analysis research, particularly the implementation of deep learning for time-series forecasting.

show abstract

Section: A Attention-based Lstmmentioning

confidence: 99%

Robust LSTM With Tuned-PSO and Bifold-Attention Mechanism for Analyzing Multivariate Time-Series

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In other words, this operation is used to generate the predicted data for the next moment using historical data with a given interval. The specific operation of the data sliding window is shown in Figure 2 [ 64 ]. Given any time series data with length N , such as {1, 2, 3, 4, 5,…, N − 1, N }, when the sliding window size is set to L and the sliding step is 1, the N-L data sets with length L + 1 are formed.…”

Section: Time Series Data Preprocessingmentioning

confidence: 99%

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

Zhang

Chen

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The time series is a kind of complex structure data, which contains some special characteristics such as high dimension, dynamic, and high noise. Moreover, multivariate time series (MTS) has become a crucial study in data mining. The MTS utilizes the historical data to forecast its variation trend and has turned into one of the hotspots. In the era of rapid information development and big data, accurate prediction of MTS has attracted much attention. In this paper, a novel deep learning architecture based on the encoder-decoder framework is proposed for MTS forecasting. In this architecture, firstly, the gated recurrent unit (GRU) is taken as the main unit structure of both the procedures in encoding and decoding to extract the useful successive feature information. Then, different from the existing models, the attention mechanism (AM) is introduced to exploit the importance of different historical data for reconstruction at the decoding stage. Meanwhile, feature reuse is realized by skip connections based on the residual network for alleviating the influence of previous features on data reconstruction. Finally, in order to enhance the performance and the discriminative ability of the new MTS, the convolutional structure and fully connected module are established. Furthermore, to better validate the effectiveness of MTS forecasting, extensive experiments are executed on two different types of MTS such as stock data and shared bicycle data, respectively. The experimental results adequately demonstrate the effectiveness and the feasibility of the proposed method.

show abstract

“…LSTM is a recurrent neural network used in deep learning that has become increasingly popular [62]. The major advantage of using the LSTM is that it is able to learn longterm dependencies, being able to handle nonlinear variations of the system, which is an important feature for time series forecasting [63].…”

Section: Lstmmentioning

confidence: 99%

Fault Prediction Based on Leakage Current in Contaminated Insulators Using Enhanced Time Series Forecasting Models

Neto¹,

Stefenon²,

Meyer³

et al. 2022

Preprint

View full text Add to dashboard Cite

To improve the monitoring of the electrical power grid it is necessary to evaluate the influence of contamination in relation to leakage current and its progression to a disruptive discharge. In this paper, insulators were tested in a saline chamber to simulate the increase of salt contamination on their surface, and to evaluate the supportability of these components. From the time series forecasting of the leakage current, it is possible to evaluate the development of the fault before a flashover occurs. Choosing which method to use is always a difficult task since some models may have higher computational effort. In this paper, for a complete evaluation, the long short-term memory (LSTM), group method of data handling (GMDH), adaptive neuro-fuzzy inference system (ANFIS), bootstrap aggregation (bagging), sequential learning (boosting), random subspace, and stacked generalization (stacking) ensemble learning models are analyzed. A review and comparison of these well-established methods for time series forecasting is performed. From the results of the best structure of the model, the hyperparameters are evaluated and the Wavelet transform is used to obtain an enhanced model.

show abstract

Improving time series forecasting using LSTM and attention models

Cited by 99 publications

References 42 publications

Robust LSTM With Tuned-PSO and Bifold-Attention Mechanism for Analyzing Multivariate Time-Series

Robust LSTM With Tuned-PSO and Bifold-Attention Mechanism for Analyzing Multivariate Time-Series

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

Fault Prediction Based on Leakage Current in Contaminated Insulators Using Enhanced Time Series Forecasting Models

Contact Info

Product

Resources

About