Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting

Abstract

Multivariate time series (MTS) forecasting plays an important role in the automation and optimization of intelligent applications. It is a challenging task, as we need to consider both complex intra-variable dependencies and inter-variable dependencies. Existing works only learn temporal patterns with the help of single inter-variable dependencies. However, there are multi-scale temporal patterns in many real-world MTS. Single inter-variable dependencies make the model prefer to learn one type of prominent and shared temporal patterns. In this paper, we propose a multi-scale adaptive graph neural network (MAGNN) to address the above issue. MAGNN exploits a multiscale pyramid network to preserve the underlying temporal dependencies at different time scales. Since the inter-variable dependencies may be different under distinct time scales, an adaptive graph learning module is designed to infer the scalespecific inter-variable dependencies without pre-defined priors. Given the multi-scale feature representations and scale-specific inter-variable dependencies, a multi-scale temporal graph neural network is introduced to jointly model intra-variable dependencies and inter-variable dependencies. After that, we develop a scale-wise fusion module to effectively promote the collaboration across different time scales, and automatically capture the importance of contributed temporal patterns. Experiments on four real-world datasets demonstrate that MAGNN outperforms the state-of-the-art methods across various settings.

Dense Sampling of Time Series for Forecasting

“…Hewage et al applied a temporal convolutional network (TCN) to weather forecasting [18]. To overcome the grid operation of CNN, Chen et al proposed a multi-scale adaptive graph neural network that jointly considers intra-and inter-variable dependencies [19].…”

LightCTS: A Lightweight Framework for Correlated Time Series Forecasting

“…Three main T-operator families are identified: (1) CNN-based T-operators [8,13,21,28,46,61,62,65], specifically Temporal Convolutional Networks (TCNs), that apply dilated causal convolutions to time series data; (2) RNN-based T-operators, such as long short term memory networks (LSTMs) [49] and gated recurrent unit networks (GRUs) [4,6,9,13,36], that process time series based on a recursive that adopt the attention mechanism to establish self-interactions of input time steps, enabling weighted temporal information extraction over long sequences. While all T-operator families have the same space complexity, the time complexity of the operators in the Transformer family is larger than those of the operators in the CNN and RNN families because of their large-size matrix multiplication [60].…”

“…There are roughly two S-operator families: (1) GCN-based Soperators, specifically Chebyshev GCNs [8,13,21,46] or Diffusion GCNs [36,60,62], utilize predefined or learned spatial adjacency matrices to capture high-order spatial correlations and (2) Transformerbased S-operators [20,44,60,64] cast attention operations across different time series to obtain their weighted spatial correlations. Theoretically, GCNs and Transformers incur the same space and time complexities for S-operators (see Table 2).…”

“…Previous studies [8,61] show that directly reducing the embedding size D inevitably lowers the representation capability of the model, resulting in subpar accuracy. Therefore, we propose instead to lighten the standard TCN using a grouping strategy.…”

“…Deep learning (DL) techniques have recently shown impressive CTS forecasting performance. A variety of DL modules, such as convolutional neural networks (CNNs) [8,21,28,46,61,62,65], recurrent neural networks (RNNs) [4,6,9,36,49], graph convolutional networks (GCNs) [8,21,36,42,46,60,62], and Transformers [44,60,64,70], are used to construct operators for extracting temporal features from individual time series or spatial features across correlated time series. These two categories of operators are referred to as temporal operators (T-operators) and spatial operators (S-operators) (see the categorization in Table 2).…”