LaserFlow: Efficient and Probabilistic Object Detection and Motion Forecasting

Meyer, Gregory P.; Charland, Jake; Pandey, Shreyash; Laddha, Ankit; Gautam, Shivam; Vallespi-Gonzalez, Carlos; Wellington, Carl

doi:10.1109/lra.2020.3047793

Cited by 28 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sensor Fusion Methods for Object Detection and Motion Forecasting: The majority of the sensor fusion works consider perception tasks, e.g. object detection [22,12,66,7,44,31,34,61,33,37] and motion forecasting [36,5,35,63,6,19,38]. They operate on multi-view LiDAR, e.g.…”

Section: Related Workmentioning

confidence: 99%

“…Prior works in the field of sensor fusion have mostly focused on the perception aspect of driving, e.g. 2D and 3D object detection [22,12,66,9,44,31,34,61,33,37], motion forecasting [22,36,5,35,63,6,19,38,32,9], and depth estimation [24,60,61,33]. These methods focus on learning a state representation that captures the geometric and semantic information of the 3D scene.…”

Section: Introductionmentioning

confidence: 99%

“…They operate primarily based on geometric feature projections between the image space and different LiDAR projection spaces, e.g. Bird's Eye View (BEV) [22,12,66,9,44,31,34,61,33] and Range View (RV) [39,37,22,38,9,51]. Information is typically aggregated from a local neighborhood around each feature in the projected 2D or 3D space.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Prakash¹,

Chitta²,

Geiger³

2021

Preprint

View full text Add to dashboard Cite

How should representations from complementary sensors be integrated for autonomous driving? Geometrybased sensor fusion has shown great promise for perception tasks such as object detection and motion forecasting. However, for the actual driving task, the global context of the 3D scene is key, e.g. a change in traffic light state can affect the behavior of a vehicle geometrically distant from that traffic light. Geometry alone may therefore be insufficient for effectively fusing representations in end-to-end driving models. In this work, we demonstrate that imitation learning policies based on existing sensor fusion methods under-perform in the presence of a high density of dynamic agents and complex scenarios, which require global contextual reasoning, such as handling traffic oncoming from multiple directions at uncontrolled intersections. Therefore, we propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. We experimentally validate the efficacy of our approach in urban settings involving complex scenarios using the CARLA urban driving simulator. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Prakash¹,

Chitta²,

Geiger³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This prediction is achieved using information from their current motion such as velocity, acceleration or previous trajectory, and some contextual elements about the scene. This context can take various forms, ranging from raw LiDAR point clouds [7], [8], [9], [10], [25], [26], [27] and RGB camera stream [25], [28], [29], [30] to more semantic representations including High-Definition maps [2], [5], [6], [7], [8], [11], [31], or detections of other agents and their motion information [5], [6], [12], [14]. Recent trajectory prediction models are designed to produce multiple forecasts, attempting to capture the multiplicity of possible futures [12], [32], [33].…”

Section: Related Workmentioning

confidence: 99%

“…Over the last years, the paradigm has shifted towards learning-based models [5], [6]. These models generally operate over two sources of information: (1) scene information about the agent's surroundings, e.g., LiDAR point clouds [7], [8], [9], [10] or bird-eye-view rasters [5], [6], [7], [8], [11], and (2) motion cues of the agent, e.g., its instantaneous velocity, acceleration, and yaw rate [5], [12] or its previous trajectory [6], [13], [14], [15]. But despite being trained with diverse modalities as input, we remark that, in practice, these models tend to base their predictions on only one modality: the previous dynamics of the agent.…”

Section: Introductionmentioning

confidence: 99%

Raising context awareness in motion forecasting

Ben-younes¹,

Zablocki²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning-based trajectory prediction models have encountered great success, with the promise of leveraging contextual information in addition to motion history. Yet, we find that state-of-the-art forecasting methods tend to overly rely on the agent's dynamics, failing to exploit the semantic cues provided at its input. To alleviate this issue, we introduce CAB, a motion forecasting model equipped with a training procedure designed to promote the use of semantic contextual information. We also introduce two novel metrics -dispersion and convergence-to-range -to measure the temporal consistency of successive forecasts, which we found missing in standard metrics. Our method is evaluated on the widely adopted nuScenes Prediction benchmark.

show abstract