FLOT: Scene Flow on Point Clouds Guided by Optimal Transport

Puy, Gilles; Boulch, Alexandre; Marlet, Renaud

doi:10.1007/978-3-030-58604-1_32

Cited by 127 publications

(189 citation statements)

References 32 publications

Supporting

Mentioning

188

Contrasting

Order By: Relevance

“…Mittal et al [44] and PointPWCNet [75] proposed self-supervised losses to infer the scene flow in an end-to-end manner. Finally, FLOT [51] proposed a simple correspondence-based end-to-end scene flow network. While our backbone also estimates correspondences, decomposing the scene into rigid agents provides us further higher-level scene understanding and enables test-time optimization while requiring less supervision.…”

Section: Related Workmentioning

confidence: 99%

“…In particular, applications such as self-driving and robot navigation rely upon a robust perception of dynamically changing 3D scenes. To equip autonomous agents with the ability to infer spatiotemporal geometric properties, there has recently been an increased interest in 3D scene flow as a form of lowlevel dynamic scene representation [37,67,73,51,49,54]. Scene flow is the 3D motion field of points in the scene [69] and is a generalization of 2D optical flow.…”

Section: Introductionmentioning

confidence: 99%

“…Second, accurately learning direct flow estimation necessitates dense supervision that is expensive to acquire and prone to annotation errors. As a result, many methods have resorted to training on simulated data [41,73,51], yet this comes at the price of a non-negligible domain gap. Other methods have attempted to solve the problem in a completely unsupervised manner [67,75,44], however they fail to provide competitive performance.…”

Section: Introductionmentioning

confidence: 99%

“…PointFlowNet [4] FlowNet3D [37] HPLFlowNet [22] MeteorNet [38] PointPwcNet [75] Flot [51], EgoFlow [67] JGWTF [44] PointPwcNet [75] EgoFlow [67] Ours…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Weakly Supervised Learning of Rigid 3D Scene Flow

Gojčič

Litany

Guibas

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. At the core of our method lies a deep architecture able to reason at the object-level by considering 3D scene flow in conjunction with other 3D tasks. This object level abstraction enables us to relax the requirement for dense scene flow supervision with simpler binary background segmentation mask and ego-motion annotations. Our mild supervision requirements make our method well suited for recently released massive data collections for autonomous driving, which do not contain dense scene flow annotations. As output, our model provides low-level cues like pointwise flow and higher-level cues such as holistic scene understanding at the level of rigid objects. We further propose a test-time optimization refining the predicted rigid scene flow. We showcase the effectiveness and generalization capacity of our method on four different autonomous driving datasets. We release our source code and pre-trained models under github.com/zgojcic/Rigid3DSceneFlow.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…PointFlowNet [4] FlowNet3D [37] HPLFlowNet [22] MeteorNet [38] PointPwcNet [75] Flot [51], EgoFlow [67] JGWTF [44] PointPwcNet [75] EgoFlow [67] Ours…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Weakly Supervised Learning of Rigid 3D Scene Flow

Gojčič

Litany

Guibas

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…3D scene flow can be applied to object detection and tracking (Behl et al, 2017;Lenz et al, 2011;Zhai et al, 2020), LiDAR odometry (Wang et al, 2021e), action recognition (Wang et al, 2017), etc. Recently, some works (Liu et al, 2019a;Wang et al, 2021d;Puy et al, 2020;Li et al, 2021b,a;Wang et al, 2021a) have been done to realize supervised estimation of 3D scene flow from two consecutive frames of point clouds. However, just like it is difficult to obtain the ground truth of optical flow (Wang et al, 2021c(Wang et al, , 2020b, the ground truth of 3D scene flow is also difficult to obtain.…”

Section: Introductionmentioning

confidence: 99%

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

Wang

Chaokang

Shen

et al. 2021

Preprint

View full text Add to dashboard Cite

3D scene flow presents the 3D motion of each point in the 3D space, which forms the fundamental 3D motion perception for autonomous driving and server robots. Although the RGBD camera or LiDAR capture discrete 3D points in space, the objects and motions usually are continuous in the macro world. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the Generative Adversarial Networks (GAN) is utilized to self-learn 3D scene flow with no need for ground truth. The fake point cloud of the second frame is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on KITTI scene flow dataset show that our method realizes promising results without ground truth. Just like a human observing a real-world scene, the proposed approach is capable of determining the consistency of the scene at different moments in spite of the exact flow value of each point is unknown in advance. Corresponding author(s) Email: wanghesheng@sjtu.edu.cn

show abstract

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

Wang

Chaokang

Shen

et al. 2021

Advanced Intelligent Systems

View full text Add to dashboard Cite

Scene flow tracks the 3D motion of each point in adjacent point clouds. It provides fundamental 3D motion perception for autonomous driving and server robot. Although red green blue depth (RGBD) camera or light detection and ranging (LiDAR) capture discrete 3D points in space, the objects and motions usually are continuous in the macroworld. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the generative adversarial networks (GAN) is utilized to self‐learn 3D scene flow without ground truth. The fake point cloud is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset show that our method realizes promising results. Just as human, the proposed method can identify the similar local structures of two adjacent frames even without knowing the ground truth scene flow. Then, the local correspondence can be correctly estimated, and further the scene flow is correctly estimated. An interactive preprint version of the article can be found here: https://www.authorea.com/doi/full/10.22541/au.163335790.03073492.

show abstract

FLOT: Scene Flow on Point Clouds Guided by Optimal Transport

Cited by 127 publications

References 32 publications

Weakly Supervised Learning of Rigid 3D Scene Flow

Weakly Supervised Learning of Rigid 3D Scene Flow

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

Contact Info

Product

Resources

About