FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

Abstract

Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. To further enhance the performance of Transformer for longterm prediction, we exploit the fact that most time series tend to have a sparse representation in well-known basis such as Fourier transform, and develop a frequency enhanced Transformer. Besides being more effective, the proposed method, termed as Frequency Enhanced Decomposed Transformer (FEDformer), is more efficient than standard Transformer with a linear complexity to the sequence length. Our empirical studies with six benchmark datasets show that compared with state-of-the-art methods, FEDformer can reduce prediction error by 14.8% and 22.6% for multivariate and univariate time series, respectively. the code will be released soon.

State Causality and Adaptive Covariance Decomposition Based Time Series Forecasting

“…FEDformer [ 5 ], being a Transformer-based model, computes attention coefficients in the frequency domain in order to represent point-wise interactions. Currently, FEDformer is the best model for long-term prediction, regardless of processing efficiency.…”

“…Thus far, the enhanced Transformer-based models for long-term forecasting include FEDformer [ 5 ], Autoformer [ 22 ], Informer [ 23 ], Reformer [ 24 ], LST [ 25 ], LogTrans [ 26 ], etc. This kind of algorithm primarily improves the attention calculation method, thus increasing the accuracy and reducing the time complexity.…”

“…However, cumulative errors appear as the major drawback in long-term forecasting for large-scale time series in the models discussed above. Transformers [ 5 , 6 ] have fixed the problem of cumulative errors to some extent by optimizing the attention calculation method and decomposition-based strategy. However, it is firmly dependent on the periodicity of the series.…”

FEDformer-Based Paddy Quality Assessment Model Affected by Toxin Change in Different Storage Environments

“…Although Transformer-based methods significantly improve the state-of-the-art in long-term series forecasting, they are not only computationally expensive but, more importantly, fail to capture a global view of the time series (the overall trend). To address these issues, FEDformer [ 45 ] combines Transformer with seasonal trend decomposition methods [ 46 ], where the seasonal trend decomposition method captures the global profile of the time series, while Transformer captures the more detailed structure. The FEDformer model combines Transformer and seasonal trend decomposition, and the main architecture uses an encoder-decoder structure with innovations.…”

Time Series Forecasting via Semi-Asymmetric Convolutional Architecture with Global Atrous Sliding Window

“…In a quite long period of time, the model based on RNN has played an important role in development of time series forecasting. Recently, transformer-based models [6][7][8][9] have been proposed enormously, which applied self-attention mechanism to distill useful semantic information in time series. However, there exists a doubt that transformer-like structure is not suitable for the task of time series forecasting.…”