Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera using Deep Residual Networks

Song, Seongjong; Shim, Hyunjung

doi:10.48550/arxiv.1809.10917

Cited by 2 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Estimating geometry of transparent objects. Works on estimating transparent object geometry are often studied in a constrained environment: For example, the work might assume a specific capturing procedure [18,24,1], known background pattern [40,19], sensor type [44] or known object 3D model [39,32,27]. Lysenkov et al [33] propose a method for the recognition and pose estimation of rigid transparent objects using a Kinect sensor.…”

Section: Related Workmentioning

confidence: 99%

ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation

Sajjan¹,

Moore²,

Pan³

et al. 2019

Preprint

View full text Add to dashboard Cite

Transparent objects are a common part of everyday life, yet they possess unique visual properties that make them incredibly difficult for standard 3D sensors to produce accurate depth estimates for. In many cases, they often appear as noisy or distorted approximations of the surfaces that lie behind them. To address these challenges, we present ClearGrasp -a deep learning approach for estimating accurate 3D geometry of transparent objects from a single RGB-D image for robotic manipulation. Given a single RGB-D image of transparent objects, ClearGrasp uses deep convolutional networks to infer surface normals, masks of transparent surfaces, and occlusion boundaries. It then uses these outputs to refine the initial depth estimates for all transparent surfaces in the scene. To train and test ClearGrasp, we construct a large-scale synthetic dataset of over 50,000 RGB-D images, as well as a real-world test benchmark with 286 RGB-D images of transparent objects and their ground truth geometries. The experiments demonstrate that ClearGrasp is substantially better than monocular depth estimation baselines and is capable of generalizing to realworld images and novel objects. We also demonstrate that ClearGrasp can be applied out-of-the-box to improve grasping algorithms' performance on transparent objects. Code, data, and benchmarks will be released. Supplementary materials: https://sites.google.com/view/cleargrasp 1

show abstract