ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals

Palazzolo, Emanuele; Behley, Jens; Lottes, Philipp; Giguère, Philippe; Stachniss, Cyrill

doi:10.1109/iros40897.2019.8967590

Cited by 125 publications

(77 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There exists also a large body of literature tackling localization and mapping changing environments, for example by filtering moving objects [13], considering residuals in matching [21], or by exploiting sequence information [33]. To achieve outdoor large-scale semantic SLAM, one can also combine 3D LiDAR sensors with RGB cameras.…”

Section: Related Workmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

378

192

View full text Add to dashboard Cite

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfelbased mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

show abstract

Section: Related Workmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

378

192

View full text Add to dashboard Cite

show abstract

“…The techniques have been extended by relaxing the rigidity assumption and several non-rigid registration techniques such as [61][62][63] aim at capturing the deformation in the object. Approaches such as [64][65][66][67][68] aim at reconstructing scenes in an online fashion either in the presence of dynamic objects or deformations. Such approaches typically operate on scans captured at a high frame rate (10-30 Hz) and thereby deal with rather small deformations in between consecutive scans.…”

Section: Plos Onementioning

confidence: 99%

Registration of spatio-temporal point clouds of plants for phenotyping

et al. 2021

Self Cite

View full text Add to dashboard Cite

Plant phenotyping is a central task in crop science and plant breeding. It involves measuring plant traits to describe the anatomy and physiology of plants and is used for deriving traits and evaluating plant performance. Traditional methods for phenotyping are often time-consuming operations involving substantial manual labor. The availability of 3D sensor data of plants obtained from laser scanners or modern depth cameras offers the potential to automate several of these phenotyping tasks. This automation can scale up the phenotyping measurements and evaluations that have to be performed to a larger number of plant samples and at a finer spatial and temporal resolution. In this paper, we investigate the problem of registering 3D point clouds of the plants over time and space. This means that we determine correspondences between point clouds of plants taken at different points in time and register them using a new, non-rigid registration approach. This approach has the potential to form the backbone for phenotyping applications aimed at tracking the traits of plants over time. The registration task involves finding data associations between measurements taken at different times while the plants grow and change their appearance, allowing 3D models taken at different points in time to be compared with each other. Registering plants over time is challenging due to its anisotropic growth, changing topology, and non-rigid motion in between the time of the measurements. Thus, we propose a novel approach that first extracts a compact representation of the plant in the form of a skeleton that encodes both topology and semantic information, and then use this skeletal structure to determine correspondences over time and drive the registration process. Through this approach, we can tackle the data association problem for the time-series point cloud data of plants effectively. We tested our approach on different datasets acquired over time and successfully registered the 3D plant point clouds recorded with a laser scanner. We demonstrate that our method allows for developing systems for automated temporal plant-trait analysis by tracking plant traits at an organ level.

show abstract

“…However, the method removed too many dynamic feature points, and the static feature points were prone to be insufficient. Palazzolo [9] presented re-fusion which uses an efficient direct tracking on the truncated signed-distance function (TSDF) and leverage color information encoded in the TSDF to estimate the pose of the sensor. For detecting dynamics, they exploited the residuals obtained after an initial registration, together with the explicit modeling of the free space in the model.…”

Section: Slam In Dynamic Scenesmentioning

confidence: 99%

OFM-SLAM: A Visual Semantic SLAM for Dynamic Indoor Environments

Zhao

Zuo

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Most of the current visual Simultaneous Localization and Mapping (SLAM) algorithms are designed based on the assumption of a static environment, and their robustness and accuracy in the dynamic environment do not behave well. The reason is that moving objects in the scene will cause the mismatch of features in the pose estimation process, which further affects its positioning and mapping accuracy. In the meantime, the three-dimensional semantic map plays a key role in mobile robot navigation, path planning, and other tasks. In this paper, we present OFM-SLAM: Optical Flow combining MASK-RCNN SLAM, a novel visual SLAM for semantic mapping in dynamic indoor environments. Firstly, we use the Mask-RCNN network to detect potential moving objects which can generate masks of dynamic objects. Secondly, an optical flow method is adopted to detect dynamic feature points. Then, we combine the optical flow method and the MASK-RCNN for full dynamic points’ culling, and the SLAM system is able to track without these dynamic points. Finally, the semantic labels obtained from MASK-RCNN are mapped to the point cloud for generating a three-dimensional semantic map that only contains the static parts of the scenes and their semantic information. We evaluate our system in public TUM datasets. The results of our experiments demonstrate that our system is more effective in dynamic scenarios, and the OFM-SLAM can estimate the camera pose more accurately and acquire a more precise localization in the high dynamic environment.

show abstract

ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals

Cited by 125 publications

References 22 publications

SuMa++: Efficient LiDAR-based Semantic SLAM

SuMa++: Efficient LiDAR-based Semantic SLAM

Registration of spatio-temporal point clouds of plants for phenotyping

OFM-SLAM: A Visual Semantic SLAM for Dynamic Indoor Environments

Contact Info

Product

Resources

About