Dynamic 3D Scene Analysis by Point Cloud Accumulation

Huang, Shengyu; Gojčič, Žan; Huang, Jiahui; Schindler, Konrad

doi:10.1007/978-3-031-19839-7_39

Cited by 22 publications

(7 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Huang et al. [HGH*22] comprehensively study point cloud registration, motion segmentation, instance segmentation, and piece‐wise rigid scene flow estimation. To this end, it is necessary to separate individual moving objects from the static background and infer their temporal and spatial properties.…”

Section: Methodsmentioning

confidence: 99%

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Aiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state‐of‐the‐art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher‐level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Xiang

Chen

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…For example, Lim et al [22] identify temporarily occluded regions in an accumulated point cloud map based on height discrepancy between query and map. Instead of removing dynamic points, Huang et al [17] explicitly target the reconstruction of moving objects for 3D scene analysis. To deal with the sparse measurements from moving objects, the authors register multiple point clouds and estimate offset vectors of previously classified moving points.…”

Section: Related Workmentioning

confidence: 99%

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Mersch

Chen

Vizzo

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

A key challenge for autonomous vehicles is to navigate in unseen dynamic environments. Separating moving objects from static ones is essential for navigation, pose estimation, and understanding how other traffic participants are likely to move in the near future. In this work, we tackle the problem of distinguishing 3D LiDAR points that belong to currently moving objects, like walking pedestrians or driving cars, from points that are obtained from non-moving objects, like walls but also parked cars. Our approach takes a sequence of observed LiDAR scans and turns them into a voxelized sparse 4D point cloud. We apply computationally efficient sparse 4D convolutions to jointly extract spatial and temporal features and predict moving object confidence scores for all points in the sequence. We develop a receding horizon strategy that allows us to predict moving objects online and to refine predictions on the go based on new observations. We use a binary Bayes filter to recursively integrate new predictions of a scan resulting in more robust estimation. We evaluate our approach on the SemanticKITTI moving object segmentation challenge and show more accurate predictions than existing methods. Since our approach only operates on the geometric information of point clouds over time, it generalizes well to new, unseen environments, which we evaluate on the Apollo dataset. Index Terms-Semantic Scene Understanding; Deep Learning Methods I. INTRODUCTION D ISTINGUISHING moving from static objects in 3D LiDAR data is a crucial task for autonomous systems and required for planning collision-free trajectories and navigating safely in dynamic environments. Moving object segmentation (MOS) can improve localization [5], [7], planning [34], mapping [5], scene flow estimation [2], [15], [37], or the prediction of future states [38], [40]. There are mapping approaches that identify if observed points are potentially moving or have moved throughout the mapping process [1], [7], [16], [28]. On the contrary, identifying objects that are actually moving within a short time horizon are of interest for online navigation [34], can improve scene flow estimation between two consecutive point clouds [2], [15], [37], or support predicting a future state of the environment [40].

show abstract

“…This allows us to apply the pseudo labels as supervision. (4) The performance gap between pseudo labels and network predictions is gradually reduced.…”

Section: Analysis On Pseudo Labelsmentioning

confidence: 99%

“…Scene flow estimation [1] aims to generate a 3D motion field of a dynamic scene. As a fundamental representation of dynamics, scene flow can be applied in various tasks, such as motion segmentation [2], 3d object detection [3], and point cloud accumulation [4], as well as multiple downstream applications including robotics and autonomous driving [5], [6]. In recent years, with the widespread application of 3D sensors and the rise of deep learning techniques for point cloud processing, learning scene flow directly from 3D point clouds has attracted increasing research attention.…”

Section: Introductionmentioning

confidence: 99%

RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Zhang

Lin

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of the rigid motion of these individual parts. Building upon this observation, we propose to generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation, in which the source point cloud is decomposed into local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels. To mitigate the impact of potential outliers on label generation, when solving the rigid registration for each region, we alternately perform three steps: establishing point correspondences, measuring the confidence for the correspondences, and updating the rigid transformation based on the correspondences and their confidence. As a result, confident correspondences will dominate label generation and a validity mask will be derived for the generated pseudo labels. By using the pseudo labels together with their validity mask for supervision, models can be trained in a self-supervised manner. Extensive experiments on FlyingThings3D and KITTI datasets demonstrate that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even performing better than some supervised counterparts. Additionally, our method is further extended to class-agnostic motion prediction and significantly outperforms previous state-of-the-art self-supervised methods on nuScenes dataset.

show abstract

Dynamic 3D Scene Analysis by Point Cloud Accumulation

Cited by 22 publications

References 49 publications

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Contact Info

Product

Resources

About