Transparent object detection and reconstruction on a mobile platform

Abstract

In this paper we propose a novel approach to detect and reconstruct transparent objects. This approach makes use of the fact that many transparent objects, especially the ones consisting of usual glass, absorb light in certain wavelengths [1]. Given a controlled illumination, this absorption is measurable in the intensity response by comparison to the background. We show the usage of a standard infrared emitter and the intensity sensor of a time of flight (ToF) camera to reconstruct the structure given we have a second view point. The structure can not be measured by the usual 3D measurements of the ToF camera.We take advantage of this fact by deriving this internal sensory contradiction from two ToF images and reconstruct an approximated surface of the original transparent object. Therefor we are using a perspectively invariant matching in the intensity channels from the first to the second view of initially acquired candidates. For each matched pixel in the first view a 3D movement can be predicted given their original 3D measurement and the known distance to the second camera position. If their line of sight did not pass a transparent object or suffered any other major defect, this prediction will highly correspond to the actual measured 3D points of the second view. Otherwise, if a detectable error occurs, we approximate a more exact point to point matching and reconstruct the original shape by triangulating the points in the stereo setup. We tested our approach using a mobile platform with one Swissranger SR4k. As this platform is mobile, we were able to create a stereo setup by moving it. Our results show a detection of transparent objects on tables while simultaneously identifying opaque objects that also existed in the test setup. The viability of our results is demonstrated by a successful automated manipulation of the respective transparent object.

Recognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor

“…A robot is able to grasp 80% of known transparent objects with the proposed algorithm and this result is robust across nonspecular backgrounds behind the objects. Our approach and other existing algorithms for pose estimation (Klank et al [12], Phillips et al [22]) can not handle overlapping transparent objects so this is the main direction for future work.…”

“…Our main focus is accurate pose estimation of transparent objects which is used to enable robotic grasping. Previous works by Klank et al [12], Phillips et al [22] are the most relevant here because other papers don't deal with pose estimation of transparent objects. These works require two views of a test scene, in contrary we use a single image of Kinect at the test stage to recognize and estimate pose of transparent objects.…”

“…Klank et al [12] detect unknown transparent objects and reconstruct them using two views of a test scene from a ToF camera. The problem is challenging especially because no training is involved and a 3D model of an object is reconstructed directly from test data.…”

A novel stereoscopic cue for figure-ground segregation of semi-transparent objects

“…Others have eschewed analytic models and instead proposed methods based on learning from an examplar set [7,19,18,10,16,13]. Additional key distinctions between approaches include whether the sensor used is passive (e.g., CCD/CMOS camera) or active (e.g., timeof-flight sensor [12]) and whether a single view or multiple views are considered.…”

I spy with my little eye: Learning optimal filters for cross-modal stereo under projected patterns

“…[3] learns object models from data and [8] detect transparent objects employing a second time of flight camera. For increasing robustness of visual category recognition across different sources such as from the web, high quality DSLRs or webcams [12] the metric learning formulation proved to be a successful.…”