3D Object Discovery and Modeling Using Single RGB-D Images Containing Multiple Object Instances

Abstract

Unsupervised object modeling is important in robotics, especially for handling a large set of objects. We present a method for unsupervised 3D object discovery, reconstruction, and localization that exploits multiple instances of an identical object contained in a single RGB-D image. The proposed method does not rely on segmentation, scene knowledge, or user input, and thus is easily scalable. Our method aims to find recurrent patterns in a single RGB-D image by utilizing appearance and geometry of the salient regions. We extract keypoints and match them in pairs based on their descriptors. We then generate triplets of the keypoints matching with each other using several geometric criteria to minimize false matches. The relative poses of the matched triplets are computed and clustered to discover sets of triplet pairs with similar relative poses. Triplets belonging to the same set are likely to belong to the same object and are used to construct an initial object model. Detection of remaining instances with the initial object model using RANSAC allows to further expand and refine the model. The automatically generated object models are both compact and descriptive. We show quantitative and qualitative results on RGB-D images with various objects including some from the Amazon Picking Challenge. We also demonstrate the use of our method in an object picking scenario with a robotic arm.

Detecting and Grouping Identical Objects for Region Proposal and Classification

“…, (1) with tp i , f p i , and f n i the true positives, false positives, and false negatives, respectively. These were summed over all C = 1000 classes.…”

“…In this work, we exploit such structure by using a multiinstance object discovery algorithm [1] that is able to discover, localize, and identify object instances that occur in a scene multiple times. The algorithm uses an RGB-D image as the input and searches for patterns of local features that occur in multiple objects.…”

Detection, Tracking and 3D Modeling of Objects with Sparse RGB-D SLAM and Interactive Perception

“…Namely, if two different segments are associated to the same map with different poses, this might be an indication of multiple instances. Reasoning geometrically to detect and track multiple object instances has been done in previous work [26], [31]. However, in the context of our system, additional care must be taken to robustly prevent map contamination in scenarios with multiple object instances.…”

Fusing Saliency Maps with Region Proposals for Unsupervised Object Localization

“…The difference between the approaches comes from the context where the solution is applied. Some work [8], [9], [10], [11], [12] focus on real robot scenarios with the end goal of locally segmenting objects. While others [13], [14], [15], [16], [17] focus on image collections or video sequences and segment the images globally.…”