TrafficBERT: Pre-trained model with large-scale data for long-range traffic flow forecasting

Jin, Kyohoon; Wi, JeongA; Lee, EunJu; Kang, Shin-Jin; Kim, Soo-Kyun; Kim, Youngbin

doi:10.1016/j.eswa.2021.115738

Cited by 30 publications

(15 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The parameter setup followed the described information given in the previous section. The performance of the proposed model was also compared against another state-of-the-art model: the TrafficBERT model (Jin et al, 2021). The used datasets were the same, but the volume was slightly less than the one used in this study.…”

Section: Resultsmentioning

confidence: 99%

“…The authors used only 61/153 days of data from the year 2012; however, here, 6 months of data from the year 2021 were used. In the referred work, 12 observed data points were used to forecast the next 12 time steps (60 minutes only) and a MAPE of 7.72% was obtained (Jin et al, 2021).…”

Section: Resultsmentioning

confidence: 99%

“…However, the effectiveness of these models is limited to a particular road; therefore, information from the adjacent roads or weather data is essential to employ them in any region or section. (Jin et al, 2021) proposed a pre-trained BERT to solve such drawbacks by achieving a MAPE of 7.72% on PeMS datasets. The model suffered from the over-fitting problem (even more than the original BERT (Devlin et al, 2018))…”

Section: Attention Based Transformermentioning

confidence: 99%

“…Transformer-based models proposed by (Xu et al, 2020;Yan et al, 2021 andJin et al, 2021) The main contributions of the current study are as follows:…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks

Reza

Ferreira

Machado

et al. 2022

Expert Systems with Applications

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Attention Based Transformermentioning

confidence: 99%

“…Transformer-based models proposed by (Xu et al, 2020;Yan et al, 2021 andJin et al, 2021) The main contributions of the current study are as follows:…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks

Reza

Ferreira

Machado

et al. 2022

Expert Systems with Applications

View full text Add to dashboard Cite

“…Traffic flow is an important indicator of urban development and operation status. Predicting, adjusting and controlling traffic flow is of great significance to urban management 1 . How to effectively predict traffic flow trends for early intervention is one of the key issues in decision‐making for traffic management 2 .…”

Section: Introductionmentioning

confidence: 99%

Short‐term traffic flow prediction based on whale optimization algorithm optimized BiLSTM_Attention

Liu

Zhao

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

With the growths in population and vehicles, traffic flow becomes more complex and uncertain disruptions occur more often. Accurate prediction of urban traffic flow is important for intelligent decision-making and warning, however, remains a challenge. Many researchers have applied neural network methods, such as convolutional neural networks and recurrent neural networks, for traffic flow prediction modeling, but training the conventional network that can obtain the best network parameters and structure is difficult, different hyperparameters lead to different network structures. Therefore, this article proposes a traffic flow prediction model based on the whale optimization algorithm (WOA) optimized BiLSTM_Attention structure to solve this problem. The traffic flow is predicted first using the BiLSTM_Attention network which is then optimized by using the WOA to obtain its four best parameters, including the learning rate, the training times, and the numbers of the nodes of two hidden layers. Finally, the four best parameters are used to build a WOA_BiLSTM_Attention model. The proposed model is compared with both conventional neural network model and neural network model optimized by the WOA. Based on the evaluation metrics of MAPE, RMSE, MAE, and R2, the WOA_BiLSTM_Attention model proposed in this article presents the best performance.

show abstract

Deep learning methods for atmospheric PM2.5 prediction: A comparative study of transformer and CNN-LSTM-attention

Cui

Liu

et al. 2023

Atmospheric Pollution Research

View full text Add to dashboard Cite

TrafficBERT: Pre-trained model with large-scale data for long-range traffic flow forecasting

Cited by 30 publications

References 20 publications

A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks

A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks

Short‐term traffic flow prediction based on whale optimization algorithm optimized BiLSTM_Attention

Deep learning methods for atmospheric PM2.5 prediction: A comparative study of transformer and CNN-LSTM-attention

Contact Info

Product

Resources

About