Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Abstract

A key challenge for autonomous vehicles is to navigate in unseen dynamic environments. Separating moving objects from static ones is essential for navigation, pose estimation, and understanding how other traffic participants are likely to move in the near future. In this work, we tackle the problem of distinguishing 3D LiDAR points that belong to currently moving objects, like walking pedestrians or driving cars, from points that are obtained from non-moving objects, like walls but also parked cars. Our approach takes a sequence of observed LiDAR scans and turns them into a voxelized sparse 4D point cloud. We apply computationally efficient sparse 4D convolutions to jointly extract spatial and temporal features and predict moving object confidence scores for all points in the sequence. We develop a receding horizon strategy that allows us to predict moving objects online and to refine predictions on the go based on new observations. We use a binary Bayes filter to recursively integrate new predictions of a scan resulting in more robust estimation. We evaluate our approach on the SemanticKITTI moving object segmentation challenge and show more accurate predictions than existing methods. Since our approach only operates on the geometric information of point clouds over time, it generalizes well to new, unseen environments, which we evaluate on the Apollo dataset. Index Terms-Semantic Scene Understanding; Deep Learning Methods I. INTRODUCTION D ISTINGUISHING moving from static objects in 3D LiDAR data is a crucial task for autonomous systems and required for planning collision-free trajectories and navigating safely in dynamic environments. Moving object segmentation (MOS) can improve localization [5], [7], planning [34], mapping [5], scene flow estimation [2], [15], [37], or the prediction of future states [38], [40]. There are mapping approaches that identify if observed points are potentially moving or have moved throughout the mapping process [1], [7], [16], [28]. On the contrary, identifying objects that are actually moving within a short time horizon are of interest for online navigation [34], can improve scene flow estimation between two consecutive point clouds [2], [15], [37], or support predicting a future state of the environment [40].

UFOMap: An Efficient Probabilistic 3D Mapping Framework That Embraces the Unknown

“…Mersch et al [3], Sun et al [4], and Toyungyernsub et al [5] develop novel frameworks to extract features and detect dynamic points utilizing spatial and temporal information. Some of these methods use the point cloud format, while others choose to translate point clouds into different representations, such as residual images, to facilitate processing.…”

RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation

“…The methods for MOS using LiDAR data can be classified into two main categories: projected range imagesbased (Chen et al 2021;Kim, Woo, and Im 2022;Sun et al 2022;Gu et al 2022;Chen et al 2022) and point cloudbased (Mersch et al 2022;He et al 2022;Liu et al 2015;Mersch et al 2023;Wang et al 2023) methods. The former involves subtracting the range images of consecutive frames to obtain residuals, which are then used to extract spatiotemporal information.…”

“…The latter category of methods uses sparse convolution to construct a network for segmenting moving point clouds. Some recent works such as Mersch et al (2022) and He et al (2022) utilize sparse convolution to extract dynamic temporal and spatial features from original point clouds using AR-SI theory (He et al 2019). These features are then employed for MOS using sparse convolution.…”

“…The goal of MOS is to accurately distinguish between moving and static objects in a scene, which is an important component in many downstream tasks, including collision detection, path planning, and navigation. Recently, many studies have been conducted on MOS using LiDAR (Chen et al 2021;Kim, Woo, and Im 2022;Sun et al 2022;Mersch et al 2022) and image (Cheng et al 2017;Voigtlaender et al 2019;Wang et al 2019;Patil et al 2021) data, which have demonstrated promising performance. However, both LiDAR and camera sensors are susceptible to bad weather conditions, which can cause a substantial reduction in MOS performance when utilized in outdoor real-world applications.…”

Dynamic Object Separation and Removal in 3D Point Cloud Map Building

“…Learning-based methods typically involve deep neural networks and supervised training with labeled datasets. Mersch et al [9] employ sparse 4D convolutions to segment receding moving objects in 3D LiDAR data, efficiently processing spatiotemporal information using sparse convolutions. Sun et al [10] develop a novel framework for fusing spatial and temporal data from LiDAR sensors, leveraging range and residual images as input to the network.…”