RGB-to-TSDF: Direct TSDF Prediction from a Single RGB Image for Dense 3D Reconstruction

Kim, Hanjun; Moon, Jiyoun; Lee, Beom-Hee

doi:10.1109/iros40897.2019.8968566

Cited by 6 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since updating the voxels directly is required, RGBD cameras are generally used to implement TSDF map construction. Kim et al [ 88 ] predicted 3D Truncated Signed Distance Function (TSDF) voxels from a single RGB image by integrating depth prediction and TSDF generation processes seamlessly. This approach enables the generation of highly accurate TSDF maps with greater efficiency compared to TSDF conversion from depth prediction alone.…”

Section: Overview Of Single Sensor Sensing Technologiesmentioning

confidence: 99%

A Review of Sensing Technologies for Indoor Autonomous Mobile Robots

Liu,

Wang,

Xie

et al. 2024

Sensors

View full text Add to dashboard Cite

As a fundamental issue in robotics academia and industry, indoor autonomous mobile robots (AMRs) have been extensively studied. For AMRs, it is crucial to obtain information about their working environment and themselves, which can be realized through sensors and the extraction of corresponding information from the measurements of these sensors. The application of sensing technologies can enable mobile robots to perform localization, mapping, target or obstacle recognition, and motion tasks, etc. This paper reviews sensing technologies for autonomous mobile robots in indoor scenes. The benefits and potential problems of using a single sensor in application are analyzed and compared, and the basic principles and popular algorithms used in processing these sensor data are introduced. In addition, some mainstream technologies of multi-sensor fusion are introduced. Finally, this paper discusses the future development trends in the sensing technology for autonomous mobile robots in indoor scenes, as well as the challenges in the practical application environments.

show abstract

Section: Overview Of Single Sensor Sensing Technologiesmentioning

confidence: 99%

A Review of Sensing Technologies for Indoor Autonomous Mobile Robots

Liu,

Wang,

Xie

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Most of the recent outstanding works [21,22] are improvements on KinectFusion, where voxel hashing and TSDF algorithms make it possible to reconstruct aerial scenes.…”

Section: Related Workmentioning

confidence: 99%

A Scale-Adaptive Time-Efficient Depth Map Fusion Algorithm

Zhang¹,

Lai²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

TSDF (truncated signed distance function) is a classical algorithm for real-time 3D reconstruction, which is simple and parallelizable. Still, the memory consumption and cost of time consumption limit the expansion of the reconstructed scene. To solve these problems, we proposed a new depth map fusion system. We open up a dynamically adjustable memory space, the size of which can be adaptively adjusted according to the coordinate distribution interval obtained from the GMM(Gaussian Mixture Model). When the scene exceeds the voxel coverage, a loop switch is triggered, and the mode of recycling the cache space is turned on. This system has a scene-adaptive memory management mechanism interspersed with CPU-GPU cooperative operations. It can improve the time efficiency without sacrificing the reconstruction accuracy and make it possible to reconstruct high-scale changing scenes. We have created a dataset of natural aviation scenes for extensive experiments. The experimental results show that our system is suitable for any scale of scenes. In terms of time, our method improves the time to fuse each depth map frame by order of magnitude. In evaluating the quality of the reconstructed surface, we compared our reconstructed surfaces with commercial 3D software, and the results show that more than $92 \%$ of the fractions differed from Metashape's results by less than 0.6m. In general, our system is more suitable for scenes with dynamic changes in scene scale and faster task requirements, which will promote the expansion of 3D reconstruction in practical applications.

show abstract

“…It has been used as a powerful representation in its own right, as way to encode individual depth images for processing by 3D CNNs [13,33]. It has also been used a reconstruction target for 3D reconstruction from single-view RGB images [20]. In our formulation, a projective TSDF prediction acts an initial approximation of the surface geometry, which allows us to model occlusion during backprojection.…”

Section: Related Workmentioning

confidence: 99%

VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion

Stier¹,

Rich²,

Sen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent volumetric 3D reconstruction methods can produce very accurate results, with plausible geometry even for unobserved surfaces. However, they face an undesirable trade-off when it comes to multi-view fusion. They can fuse all available view information by global averaging, thus losing fine detail, or they can heuristically cluster views for local fusion, thus restricting their ability to consider all views jointly. Our key insight is that greater detail can be retained without restricting view diversity by learning a view-fusion function conditioned on camera pose and image content. We propose to learn this multi-view fusion using a transformer. To this end, we introduce VoRTX, 1 an end-to-end volumetric 3D reconstruction network using transformers for wide-baseline, multi-view feature fusion. Our model is occlusion-aware, leveraging the transformer architecture to predict an initial, projective scene geometry estimate. This estimate is used to avoid backprojecting image features through surfaces into occluded regions. We train our model on ScanNet and show that it produces better reconstructions than state-of-the-art methods. We also demonstrate generalization without any finetuning, outperforming the same state-of-the-art methods on two other datasets, TUM-RGBD and ICL-NUIM.

show abstract

RGB-to-TSDF: Direct TSDF Prediction from a Single RGB Image for Dense 3D Reconstruction

Cited by 6 publications

References 29 publications

A Review of Sensing Technologies for Indoor Autonomous Mobile Robots

A Review of Sensing Technologies for Indoor Autonomous Mobile Robots

A Scale-Adaptive Time-Efficient Depth Map Fusion Algorithm

VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion

Contact Info

Product

Resources

About