FlowNet3D: Learning Scene Flow in 3D Point Clouds

Liu, Xingyu; Qi, Charles R.; Guibas, Leonidas J.

doi:10.1109/cvpr.2019.00062

Cited by 455 publications

(708 citation statements)

References 36 publications

Supporting

Mentioning

708

Contrasting

Order By: Relevance

“…Held et al utilize probabilistic approaches to point cloud segmentation and tracking [21,23,22]. Recent work demonstrates how 3D instance segmentation and 3D motion (in the form of 3D scene flow, or per-point velocity vectors) can be estimated directly on point cloud input with deep networks [59,38]. Our dataset enables 3D tracking with sensor fusion in a 360 • frame.…”

Section: Related Workmentioning

confidence: 99%

Argoverse: 3D Tracking and Forecasting With Rich Maps

Chang

Ramanan²,

Hays³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,078

788

View full text Add to dashboard Cite

Figure 1: We introduce datasets for 3D tracking and motion forecasting with rich maps for autonomous driving. Our 3D tracking dataset contains sequences of LiDAR measurements, 360 • RGB video, front-facing stereo (middle-right), and 6-dof localization. All sequences are aligned with maps containing lane center lines (magenta), driveable region (orange), and ground height. Sequences are annotated with 3D cuboid tracks (green). A wider map view is shown in the bottom-right. AbstractWe present Argoverse -two datasets designed to support autonomous vehicle machine learning tasks such as 3D tracking and motion forecasting. Argoverse was collected by a fleet of autonomous vehicles in Pittsburgh and Miami. The Argoverse 3D Tracking dataset includes 360 • images from 7 cameras with overlapping fields of view, 3D point clouds from long range LiDAR, 6-DOF pose, and 3D track annotations. Notably, it is the only modern AV dataset that provides forward-facing stereo imagery. The Argoverse Motion Forecasting dataset includes more than 300,000 5 second tracked scenarios with a particular vehicle identified for trajectory forecasting. Argoverse is the first autonomous vehicle dataset to include "HD maps" with 290 km of mapped lanes with geometric and semantic metadata. All data is released under a Creative Commons license at *Equal contribution www.argoverse.org. In our baseline experiments, we illustrate how detailed map information such as lane direction, driveable area, and ground height improves the accuracy of 3D object tracking and motion forecasting. Our tracking and forecasting experiments represent only an initial exploration of the use of rich maps in robotic perception. We hope that Argoverse will enable the research community to explore these problems in greater depth.

show abstract

Section: Related Workmentioning

confidence: 99%

Argoverse: 3D Tracking and Forecasting With Rich Maps

Chang

Ramanan²,

Hays³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,078

788

View full text Add to dashboard Cite

show abstract

“…However, its novel operator is defined on each point and pooling is the only proposed way for aggregating information. FlowNet3D [25] builds on Point-Net++ [35] and uses a flow embedding layer to mix two point clouds, so it shares the aforementioned drawbacks of [35]. Work on scene flow estimation with other input formats (stereo [19], RGBD [20], light field [27]) is less related, and we refer to Yan and Xiang [45] for a survey.…”

Section: Related Workmentioning

confidence: 99%

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

Wang

Wu³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

222

342

View full text Add to dashboard Cite

We present a novel deep neural network architecture for end-to-end scene flow estimation that directly operates on large-scale 3D point clouds. Inspired by Bilateral Convolutional Layers (BCL), we propose novel DownBCL, Up-BCL, and CorrBCL operations that restore structural information from unstructured point clouds, and fuse information from two consecutive point clouds. Operating on discrete and sparse permutohedral lattice points, our architectural design is parsimonious in computational cost. Our model can efficiently process a pair of point cloud frames at once with a maximum of 86K points per frame. Our approach achieves state-of-the-art performance on the Fly-ingThings3D and KITTI Scene Flow 2015 datasets. Moreover, trained on synthetic data, our approach shows great generalization ability on real-world data and on different point densities without fine-tuning.

show abstract

“…The difference is that the neural network processes point pairs instead of individual points. FlowNet3D [14] lets the shared neural network take mixed types of modalities, i.e. geometric features and displacement, as inputs to learn scene flow between two point clouds.…”

Section: Related Workmentioning

confidence: 99%

MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences

Liu

Yan

Bohg

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

187

153

View full text Add to dashboard Cite

Understanding dynamic 3D environment is crucial for robotic agents and many other applications. We propose a novel neural network architecture called MeteorNet for learning representations for dynamic 3D point cloud sequences. Different from previous work that adopts a gridbased representation and applies 3D or 4D convolutions, our network directly processes point clouds. We propose two ways to construct spatiotemporal neighborhoods for each point in the point cloud sequence. Information from these neighborhoods is aggregated to learn features per point. We benchmark our network on a variety of 3D recognition tasks including action recognition, semantic segmentation and scene flow estimation. MeteorNet shows stronger performance than previous grid-based methods while achieving state-of-the-art performance on Synthia. MeteorNet also outperforms previous baseline methods that are able to process at most two consecutive point clouds.To the best of our knowledge, this is the first work on deep learning for dynamic raw point cloud sequences.

show abstract

FlowNet3D: Learning Scene Flow in 3D Point Clouds

Cited by 455 publications

References 36 publications

Argoverse: 3D Tracking and Forecasting With Rich Maps

Argoverse: 3D Tracking and Forecasting With Rich Maps

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences

Contact Info

Product

Resources

About