Multi-View Shape Estimation of Transparent Containers

Abstract

The 3D localisation of an object and the estimation of its properties, such as shape and dimensions, are challenging under varying degrees of transparency and lighting conditions. In this paper, we propose a method for jointly localising container-like objects and estimating their dimensions using two wide-baseline, calibrated RGB cameras. Under the assumption of vertical circular symmetry, we estimate the dimensions of an object by sampling at different heights a set of sparse circumferences with iterative shape fitting and image re-projection to verify the sampling hypotheses in each camera using semantic segmentation masks. We evaluate the proposed method on a novel dataset of objects with different degrees of transparency and captured under different backgrounds and illumination conditions. Our method, which is based on RGB images only outperforms, in terms of localisation success and dimension estimation accuracy a deep-learning based approach that uses depth maps.

Improving Filling Level Classification with Adversarial Training

“…The task is to classify the filling level from a single RGB image. The CCM dataset [16] comprises of four views capturing under different backgrounds and illumination conditions cups and drinking glasses. The containers are transparent, translucent or opaque.…”

“…3. Sample images (resized crops) from the CORSMAL Containers Manipulation dataset [16]. Each column shows different filling types and levels, and each row shows different backgrounds and hand occlusions.…”

“…In this paper, we go beyond current approaches that use transfer learning to classify the filling level of a container [2], and evaluate transfer learning combined with adversarial training in the source domain [14,15] on the small-scale CORSMAL Containers Manipulation (CCM) [16] dataset (Fig. 1).…”

Improving filling level classification with adversarial training

“…The task is to classify the filling level from a single RGB image. The CCM dataset [16] comprises of three views capturing under different backgrounds and illumination conditions cups and drinking glasses. The containers are transparent, translucent or opaque.…”

Top-1 CORSMAL Challenge 2020 Submission: Filling Mass Estimation Using Multi-modal Observations of Human-robot Handovers

“…Different from the previous tasks, the target value is a real number. For this task, RGB-D + IR data is used to localise the object and esti-mate its dimensions [32]. The volume is then computed using a cylindrical approximation.…”