ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation

Sajjan, Shreeyak S.; Moore, Matthew; Pan, Mike; Nagaraja, Ganesh; Lee, Johnny; Zeng, Andy; Song, Shuran

doi:10.48550/arxiv.1910.02550

Cited by 6 publications

(5 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We chose the ClearPose dataset [ 16 ] to test our approach because it contains several real image scenes, in total over 350,000 annotated images, with many transparent, symmetric objects within cluttered scenes and annotated ground-truth segmentation maps and ground-truth poses. Other known available real-world datasets are ClearGrasp [ 27 ] and TransCG [ 28 ], which use plastic glasses instead of realistic real glass and include almost no glass clutter, making them uninteresting for our comparison. The complete ClearPose dataset includes several sets of scenes of different types, for example, normal objects placed jumbled on a table but also objects with filled liquors or with an additional translucent cover.…”

Section: Datasetmentioning

confidence: 99%

Transparency-Aware Segmentation of Glass Objects to Train RGB-Based Pose Estimators

Weidenbach,

Laue,

Frese

2024

Sensors

View full text Add to dashboard Cite

Robotic manipulation requires object pose knowledge for the objects of interest. In order to perform typical household chores, a robot needs to be able to estimate 6D poses for objects such as water glasses or salad bowls. This is especially difficult for glass objects, as for these, depth data are mostly disturbed, and in RGB images, occluded objects are still visible. Thus, in this paper, we propose to redefine the ground-truth for training RGB-based pose estimators in two ways: (a) we apply a transparency-aware multisegmentation, in which an image pixel can belong to more than one object, and (b) we use transparency-aware bounding boxes, which always enclose whole objects, even if parts of an object are formally occluded by another object. The latter approach ensures that the size and scale of an object remain more consistent across different images. We train our pose estimator, which was originally designed for opaque objects, with three different ground-truth types on the ClearPose dataset. Just by changing the training data to our transparency-aware segmentation, with no additional glass-specific feature changes in the estimator, the ADD-S AUC value increases by 4.3%. Such a multisegmentation can be created for every dataset that provides a 3D model of the object and its ground-truth pose.

show abstract

Section: Datasetmentioning

confidence: 99%

Transparency-Aware Segmentation of Glass Objects to Train RGB-Based Pose Estimators

Weidenbach,

Laue,

Frese

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…In Equation ( 5), µ denotes the mean, and σ denotes the variance, while σ xy denotes the cross-correlation between x and y. In addition, the constants c 1 = (255 × 0.01) 2 and c 2 = (255 × 0.03) 2 are present to ensure numerical stability [11,31].…”

Section: Metricsmentioning

confidence: 99%

“…As is well known, deep neural networks require vast amounts of (mostly labelled) data to achieve good results. Specifically, the impressive results of deep learning on challenging computer vision tasks, such as depth estimation, surface normal estimation and segmentation have leveraged (mostly free) high-quality (and sometimes synthetic) datasets [1][2][3][4]. In contrast, underwater image data is expensive to acquire due to the equipment and transportation costs involved.…”

Section: Introductionmentioning

confidence: 99%

iDehaze: Supervised Underwater Image Enhancement and Dehazing via Physically Accurate Photorealistic Simulations

2023

View full text Add to dashboard Cite

Underwater image enhancement and turbidity removal (dehazing) is a very challenging problem, not only due to the sheer variety of environments where it is applicable, but also due to the lack of high-resolution, labelled image data. In this paper, we present a novel, two-step deep learning approach for underwater image dehazing and colour correction. In iDehaze, we leverage computer graphics to physically model light propagation in underwater conditions. Specifically, we construct a three-dimensional, photorealistic simulation of underwater environments, and use them to gather a large supervised training dataset. We then train a deep convolutional neural network to remove the haze in these images, then train a second network to transform the colour space of the dehazed images onto a target domain. Experiments demonstrate that our two-step iDehaze method is substantially more effective at producing high-quality underwater images, achieving state-of-the-art performance on multiple datasets. Code, data and benchmarks will be open sourced.

show abstract

“…Training using CGI and simulation data is a promising approach for solving these limitations. 39 A major obstacle for this approach is the lack of research on generating photorealistic CGI images of chemical systems. The approach presented here can easily be used on UV and IR images, which can add another layer of information to the visible light images.…”

Section: ■ Conclusionmentioning

confidence: 99%

Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set

Eppel

Bismuth

et al. 2020

ACS Cent. Sci.

View full text Add to dashboard Cite

This work presents a machine learning approach for the computer vision-based recognition of materials inside vessels in the chemistry lab and other settings. In addition, we release a data set associated with the training of the model for further model development. The task to learn is finding the region, boundaries, and category for each material phase and vessel in an image. Handling materials inside mostly transparent containers is the main activity performed by human and robotic chemists in the laboratory. Visual recognition of vessels and their contents is essential for performing this task. Modern machine-vision methods learn recognition tasks by using data sets containing a large number of annotated images. This work presents the Vector-LabPics data set, which consists of 2187 images of materials within mostly transparent vessels in a chemistry lab and other general settings. The images are annotated for both the vessels and the individual material phases inside them, and each instance is assigned one or more classes (liquid, solid, foam, suspension, powder, ...). The fill level, labels, corks, and parts of the vessel are also annotated. Several convolutional nets for semantic and instance segmentation were trained on this data set. The trained neural networks achieved good accuracy in detecting and segmenting vessels and material phases, and in classifying liquids and solids, but relatively low accuracy in segmenting multiphase systems such as phase-separating liquids.

show abstract

ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation

Cited by 6 publications

References 52 publications

Transparency-Aware Segmentation of Glass Objects to Train RGB-Based Pose Estimators

Transparency-Aware Segmentation of Glass Objects to Train RGB-Based Pose Estimators

iDehaze: Supervised Underwater Image Enhancement and Dehazing via Physically Accurate Photorealistic Simulations

Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set

Contact Info

Product

Resources

About