Spatial-temporal ConvLSTM for vehicle driving intention prediction

Huang, Hung-Lung; Zeng, Zheni; Yao, Danya; Pei, Xin; Zhang, Yi

doi:10.26599/tst.2020.9010061

Cited by 28 publications

(18 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6. ConvLSTM implementation for HAB prediction in this paper is based on [36], [37], where the model is built using 5 ConvLSTM2D layers with batch normalization and is then followed by a Conv3D layer for spatio-temporal outputs.…”

Section: B Model Implementation Detailsmentioning

confidence: 99%

Spatiotemporal Deep-Learning-Based Algal Bloom Prediction for Lake Okeechobee Using Multisource Data Fusion

Tang

Feng

Fung

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

This study focuses on predicting harmful algal bloom (HAB) events in Lake Okeechobee, a shallow lake in Florida. A spatio-temporal deep learning model is employed to predict the levels of cyanobacteria Microcystis aeruginosa (M. aeruginosa) present in the lake for a single-day and a 14day prediction horizon. Datasets collected from remote sensing (i.e., satellite images from Jan. 2018 to Dec. 2020) and from a physics-based simulation model (i.e., daily simulation from Jan. 2018 to Dec. 2020) are available. Due to the low quality of remote sensing data caused by various environmental and technical issues, the two available datasets are fused together to create a multi-source hybrid dataset for deep learning model training. A convolutional long-short term memory (ConvLSTM) deep neural model is trained on the datasets, and the results of the predictions are compared to the true Cyanobacterial Index (CI) for that time period. Findings include 1) the deep learning model, ConvLSTM, shows promising performance for short-and mid-term HAB forecasting; and 2) the hybrid dataset that fuses remote sensing with physics-based modeling (a.k.a. modeling based on fundamental physical and biogeochemical principles) speeds up the model learning and improves its performance significantly. The proposed methodologies are reliable, and costeffective, and could be used to forecast algal bloom occurrences in shallow lakes with limited sparse observations.

show abstract

Section: B Model Implementation Detailsmentioning

confidence: 99%

Spatiotemporal Deep-Learning-Based Algal Bloom Prediction for Lake Okeechobee Using Multisource Data Fusion

Tang

Feng

Fung

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Overall, the features extracted from the convolutional layer have several dimensions; their number is reduced using the pooling layer, and finally, the fully connected layer combines all the local features and generates the final feature as the output 29 . It should be mentioned that a combination of LSTM and CNN has been adopted in many research 30 …”

Section: Research Fundamentalsmentioning

confidence: 99%

“…29 It should be mentioned that a combination of LSTM and CNN has been adopted in many research. 30 The input of a convolutional layer is usually in three-dimensional form, which includes the length, width, and the number of channels. In the first layer, inputs convolve with a set of three-dimensional filters and generate the features output map.…”

Section: Convolutional Neural Network (Cnn)mentioning

confidence: 99%

A social hybrid recommendation system using LSTM and CNN

Daneshvar

Ravanmehr

2022

Concurrency and Computation

View full text Add to dashboard Cite

With the ever-increasing use of Internet and social networks that generate a vast amount of information, there is a serious need for recommendation systems. In this article, we propose a recommender system utilizing deep neural networks that simultaneously considers both the users' ratings to the movies and the visual features of the movie poster and trailer. For this purpose, a hybrid movie recommender system, RSLC-Net, has been developed using CNN and LSTM architectures. The proposed system considers the dynamics of users' interests in the collaborative filtering engine using the LSTM network that receives user-rating sequences. In the content-based filtering engine, utilizing CNN, the visual features of movie posters and trailers are extracted, and along with the actors and the directors, similar movies are recommended to the user. Moreover, each user's social influence is calculated employing the social information available on the user's Twitter account and used in the average movie rating to improve the effectiveness of the content-based filtering part. The required datasets have been collected from MovieTweetings, Mise-en-scène, and OMDB. The evaluation results show that the accuracy and effectiveness of the proposed approach have been improved in terms of MAE and RMSE compared to the best available methods.

show abstract

“…In addition, the modeling of the influence of surrounding vehicles is also worth considering. For example, Huang et al [ 23 ] proposed a ConvLSTM-based vehicle intent prediction method for urban road scenarios. The method captured and modeled the interactions of surrounding vehicles from a bird’s-eye viewpoint, achieving approximately 60% accuracy.…”

Section: Introductionmentioning

confidence: 99%

Spatiotemporal Feature Enhancement Aids the Driving Intention Inference of Intelligent Vehicles

Chen

Líu

et al. 2022

IJERPH

View full text Add to dashboard Cite

In order that fully self-driving vehicles can be realized, it is believed that systems where the driver shares control and authority with the intelligent vehicle offer the most effective solution. An understanding of driving intention is the key to building a collaborative autonomous driving system. In this study, the proposed method incorporates the spatiotemporal features of driver behavior and forward-facing traffic scenes through a feature extraction module; the joint representation was input into an inference module for obtaining driver intentions. The feature extraction module was a two-stream structure that was designed based on a deep three-dimensional convolutional neural network. To accommodate the differences in video data inside and outside the cab, the two-stream network consists of a slow pathway that processes the driver behavior data with low frame rates, along with a fast pathway that processes traffic scene data with high frame rates. Then, a gated recurrent unit, based on a recurrent neural network, and a fully connected layer constitute an intent inference module to estimate the driver’s lane-change and turning intentions. A public dataset, Brain4Cars, was used to validate the proposed method. The results showed that compared with modeling using the data related to driver behaviors, the ability of intention inference is significantly improved after integrating traffic scene information. The overall accuracy of the intention inference of five intents was 84.92% at a time of 1 s prior to the maneuver, indicating that making full use of traffic scene information was an effective way to improve inference performance.

show abstract

Spatial-temporal ConvLSTM for vehicle driving intention prediction

Cited by 28 publications

References 23 publications

Spatiotemporal Deep-Learning-Based Algal Bloom Prediction for Lake Okeechobee Using Multisource Data Fusion

Spatiotemporal Deep-Learning-Based Algal Bloom Prediction for Lake Okeechobee Using Multisource Data Fusion

A social hybrid recommendation system using LSTM and CNN

Spatiotemporal Feature Enhancement Aids the Driving Intention Inference of Intelligent Vehicles

Contact Info

Product

Resources

About