RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecasting

Laddha, Ankit; Gautam, Shivam; Meyer, Gregory P.; Vallespi-Gonzalez, Carlos

doi:10.1109/iros51168.2021.9636083

Cited by 7 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to previous work [9,10], on nuScenes, we define our vehicle class to contain car, truck, bus, trailer and construction vehicles. On X17k, the same definition of the vehicle class is used.…”

Section: Discussionmentioning

confidence: 99%

“…The seminal work of [7] proposed a single stage end-to-end approach for predicting trajectories directly from raw lidar data using a BEV grid. This approach is further improved in subsequent works by incorporating HD map [2], reasoning about interactions [4,8], using a better network architecture [5,8] and range view based lidar representation [9,10]. As compared to multi-stage approaches, these methods are faster, easier to maintain in a production environment and have higher performance [4].…”

Section: Related Workmentioning

confidence: 99%

“…Multi-Hypothesis Trajectory Prediction: Many single stage methods produce a single deterministic trajectory [2,8] or a single distribution such as Gaussian [4] or Laplacian [9,10,5] for each waypoint of the trajectory. However, to ensure safe and efficient operations, an autonomous vehicle is required to anticipate a multitude of possible behaviors of actors in the scene.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, end-to-end approaches that directly predict trajectories from raw sensor data have gained traction [7,2,8,4,5,9,10] due to their efficiency and high performance. These approaches typically exploit lidar as well as HD maps for this task.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

Shah¹,

Huang²,

Laddha³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. Automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous radial velocity measurements. However, there are factors that make the fusion of lidar and radar information challenging, such as the relatively low angular resolution of radar measurements, their sparsity and the lack of exact time synchronization with lidar. To overcome these challenges, we propose an efficient spatio-temporal radar feature extraction scheme which achieves state-of-the-art performance on multiple large-scale datasets. Further, by incorporating radar information, we show a 52% reduction in prediction error for objects with high acceleration and a 16% reduction in prediction error for objects at longer range.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

Shah¹,

Huang²,

Laddha³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In contrast, Duerr et al [12] optimize a recurrent neural network to temporally align range image features from a single-scan semantic segmentation network. Some works project the spatial information into 2D representations like range images [5], [12], [19], [24] or BEV images [23], [26], [40] and then apply 2D or 3D convolutions to reduce the computational burden of jointly processing 4D data. Besides point-based methods [13], [14], [21] for processing point cloud sequences, representing point clouds as sparse tensors can also circumvent the backprojection issue and makes it possible to apply sparse convolutions efficiently.…”

Section: Related Workmentioning

confidence: 99%

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Mersch

Chen

Vizzo

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

A key challenge for autonomous vehicles is to navigate in unseen dynamic environments. Separating moving objects from static ones is essential for navigation, pose estimation, and understanding how other traffic participants are likely to move in the near future. In this work, we tackle the problem of distinguishing 3D LiDAR points that belong to currently moving objects, like walking pedestrians or driving cars, from points that are obtained from non-moving objects, like walls but also parked cars. Our approach takes a sequence of observed LiDAR scans and turns them into a voxelized sparse 4D point cloud. We apply computationally efficient sparse 4D convolutions to jointly extract spatial and temporal features and predict moving object confidence scores for all points in the sequence. We develop a receding horizon strategy that allows us to predict moving objects online and to refine predictions on the go based on new observations. We use a binary Bayes filter to recursively integrate new predictions of a scan resulting in more robust estimation. We evaluate our approach on the SemanticKITTI moving object segmentation challenge and show more accurate predictions than existing methods. Since our approach only operates on the geometric information of point clouds over time, it generalizes well to new, unseen environments, which we evaluate on the Apollo dataset. Index Terms-Semantic Scene Understanding; Deep Learning Methods I. INTRODUCTION D ISTINGUISHING moving from static objects in 3D LiDAR data is a crucial task for autonomous systems and required for planning collision-free trajectories and navigating safely in dynamic environments. Moving object segmentation (MOS) can improve localization [5], [7], planning [34], mapping [5], scene flow estimation [2], [15], [37], or the prediction of future states [38], [40]. There are mapping approaches that identify if observed points are potentially moving or have moved throughout the mapping process [1], [7], [16], [28]. On the contrary, identifying objects that are actually moving within a short time horizon are of interest for online navigation [34], can improve scene flow estimation between two consecutive point clouds [2], [15], [37], or support predicting a future state of the environment [40].

show abstract

Sequential Multi-view Fusion Network for Fast LiDAR Point Motion Estimation

Zhang

Wang

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecasting

Cited by 7 publications

References 27 publications

LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Sequential Multi-view Fusion Network for Fast LiDAR Point Motion Estimation

Contact Info

Product

Resources

About