Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

Xu, Mingxing; Dai, Wenrui; Liu, Chunmiao; Gao, Xing; Lin, Weiyao; Qi, Guo-Jun; Xiong, Hongkai

doi:10.48550/arxiv.2001.02908

Cited by 72 publications

(102 citation statements)

References 0 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…They have to stack more layers to learn dependency if the distance between two regions in the graph is long, and it ends up with an inefficient and oversmoothing model [31]. In recent years, Transformer [10] has been applied for traffic forecasting [11]- [14]. The canonical Transformer can be seen as a special graph neural network with a complete graph, in which any pair of regions are connected.…”

Section: Related Workmentioning

confidence: 99%

“…commendable, most works consider spatial and temporal dependency separately with independent processing modules [4]- [9], which can hardly capture their joint nature in our current setting. Some recent works apply canonical Transformer [10] to capture region dependency [11]- [14]. While impressive, canonical Transformer limits the training efficiency because it learns a region's embedding as the weighted aggregation of all the other regions based on their computed attention scores.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

Li¹,

Zhong²,

Xiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

We study the forecasting problem for traffic with dynamic, possibly periodical, and joint spatial-temporal dependency between regions. Given the aggregated inflow and outflow traffic of regions in a city from time slots 0 to t − 1, we predict the traffic at time t at any region. Prior arts in the area often consider the spatial and temporal dependencies in a decoupled manner, or are rather computationally intensive in training with a large number of hyper-parameters to tune. We propose ST-TIS, a novel, lightweight and accurate Spatial-Temporal Transformer with information fusion and region sampling for traffic forecasting. ST-TIS extends the canonical Transformer with information fusion and region sampling. The information fusion module captures the complex spatial-temporal dependency between regions. The region sampling module is to improve the efficiency and prediction accuracy, cutting the computation complexity for dependency learning fromwhere n is the number of regions. With far fewer parameters than state-of-the-art models, ST-TIS's offline training is significantly faster in terms of tuning and computation (with a reduction of up to 90% on training time and network parameters). Notwithstanding such training efficiency, extensive experiments show that ST-TIS is substantially more accurate in online prediction than state-of-the-art approaches (with an average improvement of 9.5% on RMSE, and 12.4% on MAPE compared to STDN and DSAN).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

Li¹,

Zhong²,

Xiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Similarly, Guo et al [27] proposed GATCN which deals with the spatial feature by graph attention network and the temporal feature by temporal convolutional network. Xu et al [28] proposed a spatial-temporal transformer network (STTN) composed of graph neural networks and transformer layers [29] to dynamically model various scales of spatial dependencies and capture long-range temporal dependencies. Lu et al [30] presented a temporal directed attributed graph to model complex traffic flow and then employed a message-passing mechanism and several variants of LSTMs to model spatial-temporal dependencies.…”

Section: Related Workmentioning

confidence: 99%

MAF-GNN: Multi-adaptive Spatiotemporal-flow Graph Neural Network for Traffic Speed Forecasting

Xu¹,

Liu²,

Jiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Traffic forecasting is a core element of intelligent traffic monitoring system. Approaches based on graph neural networks have been widely used in this task to effectively capture spatial and temporal dependencies of road networks. However, these approaches can not effectively define the complicated network topology. Besides, their cascade network structures have limitations in transmitting distinct features in the time and space dimensions. In this paper, we propose a Multi-adaptive Spatiotemporal-flow Graph Neural Network (MAF-GNN) for traffic speed forecasting. MAF-GNN introduces an effective Multi-adaptive Adjacency Matrices Mechanism to capture multiple latent spatial dependencies between traffic nodes. Additionally, we propose Spatiotemporal-flow Modules aiming to further enhance feature propagation in both time and space dimensions. MAF-GNN achieves better performance than other models on two real-world datasets of public traffic network, METR-LA and PeMS-Bay, demonstrating the effectiveness of the proposed approach.

show abstract

“…The power of transformer network is due to its self-attention mechanism, which can model the complex correlations between objects. Recently, some works utilized transformer based methods to extract the spatiotemporal dependencies in traffic tasks, such as trajectory prediction [33], traffic flow forecasting [32] and achieved state-of-the-art results. To our best knowledge, there is no work utilizing transformer based methods to solve the dynamic spatial correlations in OD Matrix prediction.…”

Section: 31mentioning

confidence: 99%

Completion and Augmentation based Spatiotemporal Deep Learning Approach for Short-Term Metro Origin-Destination Matrix Prediction under Limited Observable Data

Ye¹,

Zhao²,

Furong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Accurate prediction of short-term OD Matrix (i.e. the distribution of passenger flows from various origins to destinations) is a crucial task in metro systems. It is highly challenging due to the constantly changing nature of many impacting factors and the real-time delayed data collection problem. Recently, some deep learning-based models have been proposed for OD Matrix forecasting in ridehailing and high way traffic scenarios. However, these models can not sufficiently capture the complex spatiotemporal correlation between stations in metro networks due to their different prior knowledge and contextual settings. In this paper we propose a hybrid framework Multi-view TRGRU to address OD metro matrix prediction. In particular, it uses three modules to model three flow change patterns: recent trend, daily trend, weekly trend. In each module, a multi-view representation based on embedding for each station is constructed and fed into a transformer based gated recurrent structure so as to capture the dynamic spatial dependency in OD flows of different stations by a global self-attention mechanism. Extensive experiments on three large-scale, real-world metro datasets demonstrate the superiority of our Multi-view TRGRU over other competitors.

show abstract

Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

Cited by 72 publications

References 0 publications

A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

MAF-GNN: Multi-adaptive Spatiotemporal-flow Graph Neural Network for Traffic Speed Forecasting

Completion and Augmentation based Spatiotemporal Deep Learning Approach for Short-Term Metro Origin-Destination Matrix Prediction under Limited Observable Data

Contact Info

Product

Resources

About