Graph Neural Controlled Differential Equations for Traffic Forecasting

Abstract

Traffic forecasting is one of the most popular spatio-temporal tasks in the field of machine learning. A prevalent approach in the field is to combine graph convolutional networks and recurrent neural networks for the spatio-temporal processing. There has been fierce competition and many novel methods have been proposed. In this paper, we present the method of spatio-temporal graph neural controlled differential equation (STG-NCDE). Neural controlled differential equations (NCDEs) are a breakthrough concept for processing sequential data. We extend the concept and design two NCDEs: one for the temporal processing and the other for the spatial processing. After that, we combine them into a single framework. We conduct experiments with 6 benchmark datasets and 20 baselines. STG-NCDE shows the best accuracy in all cases, outperforming all those 20 baselines by non-trivial margins.

Multivariate Time Series Forecasting With Dynamic Graph Neural ODEs

“…Graphs are ubiquitous in the real world, and GNNs are designed to incorporate attributive and topological information to learn expressive node-level or graph-level representations [29], [30], where spatial correlations between nodes are explicitly modeled by passing messages from nodes' neighbors to nodes themselves. Recently, several works have emerged to tackle the traffic forecasting problem with GNN-based models [3], [8], [16], [17], [19], [20], [21], [22], [23]. Given an input multivariate time series and a predefined graph structure to characterize the static relationships between variables (i.e., nodes), they typically adopt graph convolutions to capture local spatial dependencies and use RNNs [16], [22], or 1D convolutions [3], [8], [19] to model temporal dynamics.…”

“…Given an input multivariate time series and a predefined graph structure to characterize the static relationships between variables (i.e., nodes), they typically adopt graph convolutions to capture local spatial dependencies and use RNNs [16], [22], or 1D convolutions [3], [8], [19] to model temporal dynamics. Although minor works exist to alleviate the reliance on graph priors [19], [22], [23] or conduct deeper graph propagation [21] to capture longrange spatial dependencies, they fail to completely address all three above challenges to effectively and efficiently learn stable and precise spatial-temporal dynamics on arbitrary multivariate time series data in the latent space. To bridge the gaps, we propose a simpler model by elegantly coupling two proposed continuous mechanisms, demonstrating significantly better effectiveness and efficiency.…”

“…On the other hand, as an extension of NODEs, Neural Controlled Differential Equations (NCDEs) [24] emerges as a continuous generalization of RNNs to learn on time series data naturally. A recently proposed method, STG-NCDE [23], further extends this idea to model traffic data with two different NCDEs to severally model temporal and spatial dependencies, showing good forecasting results. Although STG-NCDE learns continuous latent dynamics without relying on predefined graph priors, it cannot effectively and efficiently handle long input series due to its recursive nature like in RNNs and resource-intensive interpolating preprocessing.…”

“…Other methods, such as GTS [22] and MTGNN [19], get rid of the predefined graph structures, but they leave the first and second limitations unsolved. While a recently proposed method, STG-NCDE [23], addresses the first and third challenges with Neural Controlled Differential Equations (NCDEs) [24] and graph structure learning, it remains complex and less effective when modeling long input series because of its recursive nature and interpolating preprocessing.…”

Temporal Super-Resolution Traffic Flow Forecasting via Continuous-Time Network Dynamics

“…Recently, some studies introduce Neural ODEs to to obtain spatialtemporal hidden states with continuous depth, thus greatly improving the representation ability of the model [30,[35][36][37]. Nonetheless, on the one hand, in technique, these studies do not introduce the view of system dynamics to associate the continuity with continuous physical time; on the other hand, in application, few studies pay attention to those actually happened but unrecorded information within recording intervals, and none of these studies focus on the significant but under-valued temporal super-resolution forecasting task.…”