Long-Term Forecasting Using MAMTF: A Matrix Attention Model Based on the Time and Frequency Domains

Abstract

There are many time series forecasting methods, but there are few research methods for long-term multivariate time series forecasting, which are mainly dominated by a series of forecasting models developed on the basis of a transformer. The aim of this study is to perform forecasting for multivariate time series data and to improve the forecasting accuracy of the model. In the recent past, it has appeared that the prediction effect of linear models surpasses that of the family of self-attention mechanism models, which encourages us to look for new methods to solve the problem of long-term multivariate time series forecasting. In order to overcome the problem that the temporal order of information is easily broken in the self-attention family and that it is difficult to capture information on long-distance data using recurrent neural network models, we propose a matrix attention mechanism, which is able to weight each previous data point equally without breaking the temporal order of the data, so that the overall data information can be fully utilized. We used the matrix attention mechanism as the basic module to construct the frequency domain block and time domain block. Since complex and variable seasonal component features are difficult to capture in the time domain, mapping them to the frequency domain reduces the complexity of the seasonal components themselves and facilitates data feature extraction. Therefore, we use the frequency domain block to extract the seasonal information with high randomness and poor regularity to help the model capture the local dynamics. The time domain block is used to extract the smooth floating trend component information to help the model capture long-term change patterns. This also improves the overall prediction performance of the model. It is experimentally demonstrated that our model achieves the best prediction results on three public datasets and one private dataset.