HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects

Abstract

Among the most important prerequisites for creating and evaluating 6D object pose detectors are datasets with labeled 6D poses. With the advent of deep learning, demand for such datasets is growing continuously. Despite the fact that some of exist, they are scarce and typically have restricted setups, such as a single object per sequence, or they focus on specific object types, such as textureless industrial parts. Besides, two significant components are often ignored: training using only available 3D models instead of real data and scalability, i.e. training one method to detect all objects rather than training one detector per object. Other challenges, such as occlusions, changing light conditions and changes in object appearance, as well precisely defined benchmarks are either not present or are scattered among different datasets.In this paper we present a dataset for 6D pose estimation that covers the above-mentioned challenges, mainly targeting training from 3D models (both textured and textureless), scalability, occlusions, and changes in light conditions and object appearance. The dataset features 33 objects (17 toy, 8 household and 8 industry-relevant objects) over 13 scenes of various difficulty. We also present a set of benchmarks to test various desired detector properties, particularly focusing on scalability with respect to the number of objects and resistance to changing light conditions, occlusions and clutter. We also set a baseline for the presented benchmarks using a state-of-the-art DPOD detector. Considering the difficulty of making such datasets, we plan to release the code allowing other researchers to extend this dataset or make their own datasets in the future.

SpineDepth: A Multi-Modal Data Collection Approach for Automatic Labelling and Intraoperative Spinal Shape Reconstruction Based on RGB-D Data

“…In T-LESS [ 25 ], this error was comparable to ours with 2.5 mm for a structured light RGB-D sensor and 5.6 mm for a ToF sensor, respectively. The same sensors were employed in HomebrewedDB [ 24 ] and comparable errors of 2.56 mm (structured light) and 9.12 mm (ToF) were reported.…”

“…This was attributed to the errors in the estimated intrinsic and extrinsic RGB-D camera parameters. Therefore, similar to the practice in RGB-D datasets reported in [ 24 , 25 ], a depth correction was deemed necessary. For each specimen and each sensor, the median z -coordinate deviation was calculated over all 90 datapoints (15 push-pins × 6 viewpoints).…”

“…However, their implementation in the medical field requires large training datasets of intraoperative RGB-D streams along with corresponding ground truth 3D shapes and poses of the target anatomy. To the best of our knowledge, existing datasets only cover natural or industrial images and objects (e.g., [ 24 , 25 ]); therefore, the objective of the herein presented study was to develop a framework that enables the acquisition of large clinical data (tailored to pedicle screw placement procedure in spinal fusion surgery) for future method development. This is achieved by recording mock up surgeries performed on cadaveric specimens in a realistic OR environment using two RGB-D sensors, while simultaneously tracking the anatomy with a co-registered high-end Optical Tracking System (OTS).…”

Diffraction-engineered holography: Beyond the coherent limits of holographic displays

“…The objects in the 3D scene were randomly sampled from publicly available datasets [40][41][42][43] and each scene was rendered by Blender to have 21 varifocal images [44]. The textures of the objects used in the training stage were randomly sampled from the CC0 texture library and the textures of the objects used in the evaluation stage were sampled from the "Benchmark for 6D Object Pose Estimation" datasets [40][41][42][43]. The colors, orientations, and intensities of the light sources were randomly sampled while the maximum intensity was restricted to prevent overexposure.…”

RobotP: A Benchmark Dataset for 6D Object Pose Estimation

“…With the development of data-driven methods designed for robotics applications [33], the importance of synthetic data has been highlighted. Recent works [8,9,[34][35][36] combine real and synthetic data to generate 3D object datasets, which render 3D object models on real backgrounds in order to produce synthesized images. YCB-Video [9] dataset is the mostly used 3D object datasets for 6D object pose estimation.…”