Rigid scene flow for 3D LiDAR scans

Dewan, Ayush; Caselitz, Tim; Tipaldi, Gian Diego; Burgard, Wolfram

doi:10.1109/iros.2016.7759282

Cited by 99 publications

(75 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ushani et al [41] present a real-time four-step algorithm, which constructs occupancy grids, filters the background, solves an energy minimization problem, and refines with a filtering framework. Unlike [11,41], our approach is end-to-end. We also learn directly from data using deep networks and have no explicit assumptions, e.g., we do not assume rigid motions.…”

Section: Related Workmentioning

confidence: 99%

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

Wang

Wu³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

206

342

View full text Add to dashboard Cite

We present a novel deep neural network architecture for end-to-end scene flow estimation that directly operates on large-scale 3D point clouds. Inspired by Bilateral Convolutional Layers (BCL), we propose novel DownBCL, Up-BCL, and CorrBCL operations that restore structural information from unstructured point clouds, and fuse information from two consecutive point clouds. Operating on discrete and sparse permutohedral lattice points, our architectural design is parsimonious in computational cost. Our model can efficiently process a pair of point cloud frames at once with a maximum of 86K points per frame. Our approach achieves state-of-the-art performance on the Fly-ingThings3D and KITTI Scene Flow 2015 datasets. Moreover, trained on synthetic data, our approach shows great generalization ability on real-world data and on different point densities without fine-tuning.

show abstract

Section: Related Workmentioning

confidence: 99%

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

Wang

Wu³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

206

342

View full text Add to dashboard Cite

show abstract

“…We compare our method to four baselines: a point cloudbased method using a CRF [8], two point-matching methods, and an Iterative Closest Point [4] (ICP) baseline.…”

Section: Baseline Methodsmentioning

confidence: 99%

PointFlowNet: Learning Representations for Rigid Motion Estimation From Point Clouds

Behl

Paschalidou

Donné

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

112

View full text Add to dashboard Cite

Despite significant progress in image-based 3D scene flow estimation, the performance of such approaches has not yet reached the fidelity required by many applications. Simultaneously, these applications are often not restricted to image-based estimation: laser scanners provide a popular alternative to traditional cameras, for example in the context of self-driving cars, as they directly yield a 3D point cloud. In this paper, we propose to estimate 3D motion from such unstructured point clouds using a deep neural network. In a single forward pass, our model jointly predicts 3D scene flow as well as the 3D bounding box and rigid body motion of objects in the scene. While the prospect of estimating 3D scene flow from unstructured point clouds is promising, it is also a challenging task. We show that the traditional global representation of rigid body motion prohibits inference by CNNs, and propose a translation equivariant representation to circumvent this problem. For training our deep network, a large dataset is required. Because of this, we augment real scans from KITTI with virtual objects, realistically modeling occlusions and simulating sensor noise. A thorough comparison with classic and learning-based techniques highlights the robustness of the proposed approach.

show abstract

“…For our first evaluation on the KITTI dataset ( Table 4 in the main paper), we evaluate on Lidar scans with removed grounds, for two reasons. First, this is a more fair comparison with previous works that relied on ground segmentation/removal as a pre-processing step [8,26]. Second, since our model is not trained on the KITTI dataset (due to the very small size of the dataset), it is hard to make it generalize to predicting motions of ground points because the ground is a large flat piece of geometry with little cue to tell its motion.…”

Section: 2)mentioning

confidence: 99%

FlowNet3D: Learning Scene Flow in 3D Point Clouds

Liu

Guibas

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

419

676

View full text Add to dashboard Cite

Many applications in robotics and human-computer interaction can benefit from understanding 3D motion of points in a dynamic environment, widely noted as scene flow. While most previous methods focus on stereo and RGB-D images as input, few try to estimate scene flow directly from point clouds. In this work, we propose a novel deep neural network named FlowNet3D that learns scene flow from point clouds in an end-to-end fashion. Our network simultaneously learns deep hierarchical features of point clouds and flow embeddings that represent point motions, supported by two newly proposed learning layers for point sets. We evaluate the network on both challenging synthetic data from FlyingThings3D and real Lidar scans from KITTI. Trained on synthetic data only, our network successfully generalizes to real scans, outperforming various baselines and showing competitive results to the prior art. We also demonstrate two applications of our scene flow output (scan registration and motion segmentation) to show its potential wide use cases.

show abstract

Rigid scene flow for 3D LiDAR scans

Cited by 99 publications

References 15 publications

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

PointFlowNet: Learning Representations for Rigid Motion Estimation From Point Clouds

FlowNet3D: Learning Scene Flow in 3D Point Clouds

Contact Info

Product

Resources

About