ST-P3: End-to-End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning

Hu, Shengchao; Chen, Li; Wu, Penghao; Li, Hongyang; Yan, Junchi; Tao, Dacheng

doi:10.1007/978-3-031-19839-7_31

Cited by 72 publications

(30 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Modeling and predicting future trajectories play an indispensable role in various applications, i.e., autonomous driving [23,25,65], motion capture [57,59], behavior understanding [20,38], etc. However, accurately predicting movement patterns is challenging due to their complex and subtle nature.…”

Section: Introductionmentioning

confidence: 99%

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

Xu¹,

Bazarjani²,

Chi³

et al. 2023

Preprint

View full text Add to dashboard Cite

Trajectory prediction is a crucial undertaking in understanding entity movement or human behavior from observed sequences. However, current methods often assume that the observed sequences are complete while ignoring the potential for missing values caused by object occlusion, scope limitation, sensor failure, etc. This limitation inevitably hinders the accuracy of trajectory prediction. To address this issue, our paper presents a unified framework, the Graph-based Conditional Variational Recurrent Neural Network (GC-VRNN), which can perform trajectory imputation and prediction simultaneously. Specifically, we introduce a novel Multi-Space Graph Neural Network (MS-GNN) that can extract spatial features from incomplete observations and leverage missing patterns. Additionally, we employ a Conditional VRNN with a specifically designed Temporal Decay (TD) module to capture temporal dependencies and temporal missing patterns in incomplete trajectories. The inclusion of the TD module allows for valuable information to be conveyed through the temporal flow. We also curate and benchmark three practical datasets for the joint problem of trajectory imputation and prediction. Extensive experiments verify the exceptional performance of our proposed method. As far as we know, this is the first work to address the lack of benchmarks and techniques for trajectory imputation and prediction in a unified manner.

show abstract

Section: Introductionmentioning

confidence: 99%

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

Xu¹,

Bazarjani²,

Chi³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Forecasting future scene evolutions is important to the safety of autonomous driving vehicles. Most existing methods follow a conventional pipeline of perception, prediction, and planning [17,18,25]. Perception aims to obtain a semantic understanding of the surrounding scene such as 3D object detection [19,32,33] and semantic map construction [30,34,37,66].…”

Section: Introductionmentioning

confidence: 99%

“…Perception aims to obtain a semantic understanding of the surrounding scene such as 3D object detection [19,32,33] and semantic map construction [30,34,37,66]. The subsequent prediction module captures the motion of other traffic participants [11,14,24,66], and the planning module then makes decisions based on previous outputs [17,18,25,45]. How-ever, this serial design usually requires ground-truth labels at each stage of training, yet the instance-level bounding boxes and high-definition maps are difficult to annotate.…”

Section: Introductionmentioning

confidence: 99%

GTOC 11: Results from Tsinghua University and Shanghai Institute of Satellite Engineering

Zhang

Xiang

et al. 2023

Acta Astronautica

View full text Add to dashboard Cite

“…R ECENT advancements in deep learning have significantly transformed the field of AD [10], [11], [21], [22]. The traditional pipeline of AD systems is structured around three core stages: perception, prediction, and planning [20].…”

Section: Introductionmentioning

confidence: 99%

Estimation of 6D Object Pose Using a 2D Bounding Box

Hong

Liu

Jahangir

et al. 2021

Sensors

View full text Add to dashboard Cite

This paper provides an efficient way of addressing the problem of detecting or estimating the 6-Dimensional (6D) pose of objects from an RGB image. A quaternion is used to define an object′s three-dimensional pose, but the pose represented by q and the pose represented by -q are equivalent, and the L2 loss between them is very large. Therefore, we define a new quaternion pose loss function to solve this problem. Based on this, we designed a new convolutional neural network named Q-Net to estimate an object’s pose. Considering that the quaternion′s output is a unit vector, a normalization layer is added in Q-Net to hold the output of pose on a four-dimensional unit sphere. We propose a new algorithm, called the Bounding Box Equation, to obtain 3D translation quickly and effectively from 2D bounding boxes. The algorithm uses an entirely new way of assessing the 3D rotation (R) and 3D translation rotation (t) in only one RGB image. This method can upgrade any traditional 2D-box prediction algorithm to a 3D prediction model. We evaluated our model using the LineMod dataset, and experiments have shown that our methodology is more acceptable and efficient in terms of L2 loss and computational time.

show abstract

ST-P3: End-to-End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning

Cited by 72 publications

References 41 publications

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

GTOC 11: Results from Tsinghua University and Shanghai Institute of Satellite Engineering

Estimation of 6D Object Pose Using a 2D Bounding Box

Contact Info

Product

Resources

About