Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Song, Seongjong; Shim, Hyunjung

doi:10.1007/978-3-030-20873-8_41

Cited by 7 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Works in this area generally fall in four main schemes: constraints driven, monocular depth estimation, depth completion from sparse point cloud, and depth completion given noisy RGB-D. Our work falls in the last scheme. Constraint driven approaches assume a specific setup method with fixed viewpoint(s) and capturing procedure [12,13,14,15,16], sensor type [17,18,11], or known object models [19,20,21]. Our proposed depth completion method does not apply any assumptions.…”

Section: Related Workmentioning

confidence: 99%

Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects

Xu,

Wang,

Eppel

et al. 2021

Preprint

View full text Add to dashboard Cite

The basis of many object manipulation algorithms is RGB-D input. Yet, commodity RGB-D sensors can only provide distorted depth maps for a wide range of transparent objects due light refraction and absorption. To tackle the perception challenges posed by transparent objects, we propose TranspareNet, a joint point cloud and depth completion method, with the ability to complete the depth of transparent objects in cluttered and complex scenes, even with partially filled fluid contents within the vessels. To address the shortcomings of existing transparent object data collection schemes in literature, we also propose an automated dataset creation workflow that consists of robot-controlled image collection and vision-based automatic annotation. Through this automated workflow, we created Toronto Transparent Objects Depth Dataset (TODD), which consists of nearly 15000 RGB-D images. Our experimental evaluation demonstrates that TranspareNet outperforms existing state-of-the-art depth completion methods on multiple datasets, including Clear-Grasp, and that it also handles cluttered scenes when trained on TODD. Code and dataset will be released at https://www.pair.toronto.edu/TranspareNet/

show abstract

Section: Related Workmentioning

confidence: 99%

Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects

Xu,

Wang,

Eppel

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Further, several works on improving on various aspects and applications using machine learning followed, e.g. online calibration using RGB information [26], frame rate optimization [5], power efficiency [6], robotic arm setups [27] or translucent materials [28], to name a few. The aforementioned approaches all use standard 2D CNNs and thus consider the denoising problem as an image task.…”

Section: Related Workmentioning

confidence: 99%

“…However, existing methods treat the task of ToF denoising as a 2D image problem and do not take into account the explicit 3D information in their computations. In these works, the depth information is usually used as an input to standard Convolutional Neural Networks (CNN) for images [1,2,20,27,28], while the underlying 3D structure is not analyzed. In this work instead, we propose a new neural network architecture that projects the problem into the 3D domain and makes use of point convolutional neural networks [13] to analyze the noisy reconstruction and adjust the point positions along the view direction, see Fig.…”

Section: Introductionmentioning

confidence: 99%

RADU: Ray-Aligned Depth Update Convolutions for ToF Data Denoising

Schelling¹,

Hermosilla²,

Ropinski³

2021

Preprint

View full text Add to dashboard Cite

Time-of-Flight (ToF) cameras are subject to high levels of noise and distortions due to Multi-Path-Interference (MPI). While recent research showed that 2D neural networks are able to outperform previous traditional Stateof-the-Art (SOTA) methods on denoising ToF-Data, little research on learning-based approaches has been done to make direct use of the 3D information present in depth images. In this paper, we propose an iterative denoising approach operating in 3D space, that is designed to learn on 2.5D data by enabling 3D point convolutions to correct the points' positions along the view direction. As labeled real world data is scarce for this task, we further train our network with a self-training approach on unlabeled real world data to account for real world statistics. We demonstrate that our method is able to outperform SOTA methods on several datasets, including two real world datasets and a new large-scale synthetic data set introduced in this paper.

show abstract

“…In [32,33], transparent objects are reconstructed with known background patterns. In [34][35][36], the depth of transparent objects is reconstructed using time of flight camera, since glass absorbs light of certain wavelengths. 3D geometry can be reconstructed from multiple views or 3D scanning.…”

Section: Related Workmentioning

confidence: 99%

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Chen

Yao

et al. 2020

Sensors

View full text Add to dashboard Cite

6DoF object pose estimation is a foundation for many important applications, such as robotic grasping, automatic driving, and so on. However, it is very challenging to estimate 6DoF pose of transparent object which is commonly seen in our daily life, because the optical characteristics of transparent material lead to significant depth error which results in false estimation. To solve this problem, a two-stage approach is proposed to estimate 6DoF pose of transparent object from a single RGB-D image. In the first stage, the influence of the depth error is eliminated by transparent segmentation, surface normal recovering, and RANSAC plane estimation. In the second stage, an extended point-cloud representation is presented to accurately and efficiently estimate object pose. As far as we know, it is the first deep learning based approach which focuses on 6DoF pose estimation of transparent objects from a single RGB-D image. Experimental results show that the proposed approach can effectively estimate 6DoF pose of transparent object, and it out-performs the state-of-the-art baselines by a large margin.

show abstract

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Cited by 7 publications

References 32 publications

Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects

Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects

RADU: Ray-Aligned Depth Update Convolutions for ToF Data Denoising

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Contact Info

Product

Resources

About