TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

Abstract

The ability to simultaneously track and reconstruct multiple objects moving in the scene is of the utmost importance for robotic tasks such as autonomous navigation and interaction. Virtually all of the previous attempts to map multiple dynamic objects have evolved to store individual objects in separate reconstruction volumes and track the relative pose between them. While simple and intuitive, such formulation does not scale well with respect to the number of objects in the scene and introduces the need for an explicit occlusion handling strategy. In contrast, we propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. To this end, we introduce a novel multi-object TSDF formulation that can encode multiple object surfaces at any given location in the map. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation. Code is available at https://github.com/ethz-asl/tsdf-plusplus.

A comprehensive survey on non‐cooperative collision avoidance for micro aerial vehicles: Sensing and obstacle detection

“…It can achieve high reconstruction quality and can capture the complex topology of 3D objects. One recent work that used TSDF directly for obstacle representation is Grinvald et al (2021). In this work, they could store more than one value of objects' surfaces in one voxel for multi‐obstacle detection.…”

A Scale-Adaptive Time-Efficient Depth Map Fusion Algorithm

“…Our system involves only map mapping, namely solving the 3D model with known color maps, depth maps, and poses. However, the popular TSDF algorithm frameworks, such as KinectFusion [9], Kintinuous [18], c-blox [22], and TSDF++ [28], include not only map mapping but also pose estimation and other aspects, so it is difficult for us to find a suitable framework for experimental comparison. Therefore, we only made a temporal comparison with the TSDF framework [29].…”

“…After each frame of the depth map is loaded, the framework [28] needs to convert the coordinates for each voxel in the 3D voxel grid. The size of the 3D cube grid is fixed, so many voxels are involved in useless operations when each fusion module is executed, resulting in high time costs.…”

Neural Implicit Vision-Language Feature Fields

“…These methods assign a semantic class to each individual voxel in the scene. Voxel-based dense semantic representations typically operate on static scenes, but some have explored modeling dynamic objects [32], [33].…”