Friend or foe: exploiting sensor failures for transparent object localization and classification

Seib, Viktor; Barthen, Andreas; Marohn, Philipp; Paulus, Dietrich

doi:10.1117/12.2266255

Cited by 13 publications

(15 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [28], glass objects are localized by joint inferring boundary and depth. In [29][30][31], transparent objects are detected using CNN networks. However, these methods only detect transparent object, and do not estimate 6DoF object pose.…”

Section: Related Workmentioning

confidence: 99%

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Chen

Yao

et al. 2020

Sensors

View full text Add to dashboard Cite

6DoF object pose estimation is a foundation for many important applications, such as robotic grasping, automatic driving, and so on. However, it is very challenging to estimate 6DoF pose of transparent object which is commonly seen in our daily life, because the optical characteristics of transparent material lead to significant depth error which results in false estimation. To solve this problem, a two-stage approach is proposed to estimate 6DoF pose of transparent object from a single RGB-D image. In the first stage, the influence of the depth error is eliminated by transparent segmentation, surface normal recovering, and RANSAC plane estimation. In the second stage, an extended point-cloud representation is presented to accurately and efficiently estimate object pose. As far as we know, it is the first deep learning based approach which focuses on 6DoF pose estimation of transparent objects from a single RGB-D image. Experimental results show that the proposed approach can effectively estimate 6DoF pose of transparent object, and it out-performs the state-of-the-art baselines by a large margin.

show abstract

Section: Related Workmentioning

confidence: 99%

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Chen

Yao

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Most recently, owing to the rapid progress in machine learning algorithms, the correspondence problem in depth estimation is being addressed by learning based approach [15,16]. Seib et al [8] extended learning-based approach to an end-to-end framework using Convolutional Neural Network (CNN). They handled depth maps containing transparent objects, yet focusing on classifying and localizing predefined objects exploiting holes of depth image.…”

Section: Related Workmentioning

confidence: 99%

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Song

Shim

2019

Computer Vision – ACCV 2018

View full text Add to dashboard Cite

We propose a novel approach to recovering the translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording the translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with the surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt the deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

show abstract

“…Existing methods for detection of transparent materials can be classified into two categories based on the algorithm that they employ: traditional methods [2], [3], [8]- [11], [13]- [15] and leaning methods [5]- [7](e.g., deep learning based methods). In this section, we review the recent and most popular works related to transparent material detection.…”

Section: Related Workmentioning

confidence: 99%

“…In the past years, studies have been conducted for recognition of transparent materials [2]- [7]. Although some progress has been made, recognizing transparent materials is still open problem.…”

Section: Introductionmentioning

confidence: 99%

“…Although some progress has been made, recognizing transparent materials is still open problem. Most existing methods use hand-engineered features or other traditional methods as in [2], [3], [8]- [11], and few implemented learning methods to detect transparent objects [5]- [7]. The traditional methods impose several constraints and require prior information that make an inference of transparency computationally expensive and challenging.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging an Instance Segmentation Method for Detection of Transparent Materials

Madessa

Dong

et al. 2019

2019 IEEE SmartWorld, Ubiquitous Intelligence &Amp; Computing, Advanced &Amp; Trusted Computing, Scalable Computing &Amp; Commu

View full text Add to dashboard Cite

Automatic detection of transparent materials (e.g., glass, plastic, etc.) is essential in many computer vision tasks. For example, a robot could use such a system to navigate around transmissive materials or operate tasks with these materials without causing damage. Nevertheless, it is challenging task as such materials exhibit less texture or background scenes dominate visual perception. Existing methods used either handengineered or leaned features to detect and segment transparent objects. We argue that pixel-wise detection and segmentation of transmissive materials improve detection performance and provide the fine-grained information compared to detecting bounding boxes of objects (i.e., localisation task). In this paper, we leverage a robust and state-of-the-art instance segmentation method namely, Mask R-CNN, in order to detect transparent materials. To be specific, we train the model on a new dataset with an evaluation based on publicly available dataset. Experimental results show that the adopted method significantly enhances the performance of transparent material detection. In particular, the resulting binary masks provides the pixel-level information for an improved understanding and analysis of transparency.

show abstract

Friend or foe: exploiting sensor failures for transparent object localization and classification

Cited by 13 publications

References 3 publications

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Leveraging an Instance Segmentation Method for Detection of Transparent Materials

Contact Info

Product

Resources

About