LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

Abstract

In this paper, we present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. Automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous radial velocity measurements. However, there are factors that make the fusion of lidar and radar information challenging, such as the relatively low angular resolution of radar measurements, their sparsity and the lack of exact time synchronization with lidar. To overcome these challenges, we propose an efficient spatio-temporal radar feature extraction scheme which achieves state-of-the-art performance on multiple large-scale datasets. Further, by incorporating radar information, we show a 52% reduction in prediction error for objects with high acceleration and a 16% reduction in prediction error for objects at longer range.

MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data

“…This demonstrates that our proposed method can leverage multiple views much more effectively than previous multi-view end-to-end methods. Finally, we show that our method, with only LiDAR information, is able to outperform multi-sensor methods like LiRANet [5] (which uses RADAR in addition to LiDAR) and LC-MV [10] (which uses camera images in addition to LiDAR).…”

“…These methods show that recent work on multi-modal predictions and the use of interaction graphs to model complex relationships can be easily extended to the framework of joint object detection and motion forecasting. [5] and [10] are recent multi-sensor methods that build on top of [4] by using radar and camera inputs respectively. These methods, by virtue of operating in BEV, lose out on high-resolution point information and are often limited by range of operation.…”

“…Similarly to [4,5]. we train the proposed method endto-end using a multi-task loss incorporating both detection and trajectory loss:…”

“…56.1 143 ----RVFuseNet [3] 59.9 120 ----LiRANet [5] 63. motion forecasting when we compare our method to the best RV-based method RVFuseNet [3], and the state-of-the-art BEV-based method MultiXNet [4]. Notably, our method shows a ∼ 15% improvement on pedestrian detection, a ∼ 40% improvement on bike detection, and a ∼ 30% improvement on motion forecasting of bikes, as compared to the best BEV-only MultiXNet.…”

“…However, these methods optimize for these tasks separately, suffering from cascading errors and missing the opportunity to share learned features for both tasks [1]. To overcome these issues, multiple end-to-end methods have been proposed [2,3,4,5] for jointly solving both detection and motion forecasting. These methods have demonstrated excellent performance [1] while operating in real-time.…”