Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency

Schmid, Lukas; Delmerico, Jeffrey; Schönberger, Johannes; Nieto, Juan; Pollefeys, Marc; Siegwart, Roland; Cadena, Cesar

doi:10.48550/arxiv.2109.10165

Cited by 5 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10 shows example local maps, drawn samples, and selected NBVs from the real system. Due to the high sensor noise and localization errors, the map extracted from [3] after depth fusion is notably more noisy than in simulation. Together with the different scale of the narrow environment, this constitutes a major domain shift.…”

Section: E Robot Experimentsmentioning

confidence: 88%

“…It is equipped with an Intel Realsense D435RGB-D camera of F oV = 87 • and r = 2.5m. We use the standard TB stack for online state estimation, a custom PD controller to track trajectories in narrow environments, and the Single-TSDF implementation of [3] to extract safe local maps L t from the depth camera. All computation is performed on a laptop with an Intel i7-8550U CPU @1.80GHz.…”

Section: Methodsmentioning

confidence: 99%

“…In this paper, we consider the problem of exploring unknown environments represented by standard occupancy maps, encoding free, occupied and unknown space. This common map representation can be obtained from any volumetric mapping system [1]- [3] and enables a ground robot to plan in continuous 2D space.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning

Schmid,

Ni,

Zhong

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Exploration is a fundamental problem in robotics. While sampling-based planners have shown high performance, they are oftentimes compute intensive and can exhibit high variance. To this end, we propose to directly learn the underlying distribution of informative views based on the spatial context in the robot's map. We further explore a variety of methods to also learn the information gain. We show in thorough experimental evaluation that our proposed system improves exploration performance by up to 28% over classical methods, and find that learning the gains in addition to the sampling distribution can provide favorable performance vs. compute trade-offs for compute-constrained systems. We demonstrate in simulation and on a low-cost mobile robot that our system generalizes well to varying environments.

show abstract

Section: E Robot Experimentsmentioning

confidence: 88%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning

Schmid,

Ni,

Zhong

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The literature has focused on both object-based maps [10,14,41,42,53,56] and dense maps, including volumetric models [23,38,40], point clouds [6,32,61], and 3D meshes [48,51]. Some approaches combine objects and dense map models [31,39,54,68]. These approaches are not concerned with estimating higher-level semantics (e.g., rooms) and typically return dense models that might not be directly amenable for navigation [44].…”

Section: Related Workmentioning

confidence: 99%

Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization

Hughes¹,

Chang²,

Carlone³

2022

Preprint

View full text Add to dashboard Cite

3D scene graphs have recently emerged as a powerful high-level representation of 3D environments. A 3D scene graph describes the environment as a layered graph where nodes represent spatial concepts at multiple levels of abstraction (from low-level geometry to high-level semantics including objects, places, rooms, buildings, etc.) and edges represent relations between concepts. While 3D scene graphs can serve as an advanced "mental model" for robots, how to build such a rich representation in real-time is still uncharted territory.This paper describes the first real-time Spatial Perception engINe (SPIN), a suite of algorithms to build a 3D scene graph from sensor data in real-time. Our first contribution is to develop real-time algorithms to incrementally construct the layers of a scene graph as the robot explores the environment; these algorithms build a local Euclidean Signed Distance Function (ESDF) around the current robot location, extract a topological map of places from the ESDF, and then segment the places into rooms using an approach inspired by community-detection techniques. Our second contribution is to investigate loop closure detection and optimization in 3D scene graphs. We show that 3D scene graphs allow defining hierarchical descriptors for loop closure detection; our descriptors capture statistics across layers in the scene graph, ranging from low-level visual appearance, to summary statistics about objects and places. We then propose the first algorithm to optimize a 3D scene graph in response to loop closures; our approach relies on embedded deformation graphs to simultaneously correct all layers of the scene graph. We implement the proposed SPIN into a highly parallelized architecture, named Hydra, that combines fast early and midlevel perception processes (e.g., local mapping) with slower highlevel perception (e.g., global optimization of the scene graph). We evaluate Hydra on simulated and real data and show it is able to reconstruct 3D scene graphs with an accuracy comparable with batch offline methods, while running online.

show abstract

“…Therefore, we choose to represent the whole scene as a set of planes, each having an associated SDF volume that describes the geometric details of the objects attached to it, which we term as the PlaneSDF representation. Similiar to the idea of dividing the whole environment into submaps, e.g., based on time intervals [7] or objects [6], [9], agents could maintain multiple PlaneSDF volumes of scalable sizes in lieu of a single chunk of global SDF while saving update and memory reload time by updating volumes only in the current viewing frustum. Furthermore, this representation is also more robust to localization drift as local regional correction can be performed patch by patch each time two planes from different traverses are registered via plane pose.…”

mentioning

confidence: 99%

PlaneSDF-Based Change Detection for Long-Term Dense Mapping

Fu¹,

Lin

Taguchi

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

The ability to process environment maps across multiple sessions is critical for robots operating over extended periods of time. Specifically, it is desirable for autonomous agents to detect changes amongst maps of different sessions so as to gain a conflict-free understanding of the current environment. In this paper, we look into the problem of change detection based on a novel map representation, dubbed Plane Signed Distance Fields (PlaneSDF), where dense maps are represented as a collection of planes and their associated geometric components in SDF volumes. Given point clouds of the source and target scenes, we propose a three-step PlaneSDF-based change detection approach:(1) PlaneSDF volumes are instantiated within each scene and registered across scenes using plane poses; 2D height maps and object maps are extracted per volume via height projection and connected component analysis. ( 2) Height maps are compared and intersected with the object map to produce a 2D change location mask for changed object candidates in the source scene.(3) 3D geometric validation is performed using SDF-derived features per object candidate for change mask refinement. We evaluate our approach on both synthetic and real-world datasets and demonstrate its effectiveness via the task of changed object detection. Supplementary video: https://ieeexplore.ieee.org.

show abstract

Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency

Cited by 5 publications

References 31 publications

Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning

Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning

Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization

PlaneSDF-Based Change Detection for Long-Term Dense Mapping

Contact Info

Product

Resources

About