DPOD: 6D Pose Object Detector and Refiner

Abstract

In this paper we present a novel deep learning method for 3D object detection and 6D pose estimation from RGB images. Our method, named DPOD (Dense Pose Object Detector), estimates dense multi-class 2D-3D correspondence maps between an input image and available 3D models. Given the correspondences, a 6DoF pose is computed via PnP and RANSAC. An additional RGB pose refinement of the initial pose estimates is performed using a custom deep learning-based refinement scheme. Our results and comparison to a vast number of related works demonstrate that a large number of correspondences is beneficial for obtaining high-quality 6D poses both before and after refinement. Unlike other methods that mainly use real data for training and do not train on synthetic renderings, we perform evaluation on both synthetic and real training data demonstrating superior results before and after refinement when compared to all recent detectors. While being precise, the presented approach is still real-time capable. Recent deep learning-based approaches, such as SSD6D [15], YOLO6D [33], AAE [31], PoseCNN [34] and PVNet [25], are the current top performers for this task in RGB images. Even though they all perform evaluation on LineMOD and OCCLUSION datasets, each of them focuses on different aspects of the 6D pose estimation pipeline. The majority is trained on real data [33, 34, 25, 14] while only SSD6D [15] and AAE [31] are trained on syn-

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

“…Unlike the previous categories of methods, i.e., classification-based and regressionbased, this category performs the classification and regression tasks within a single architecture. The methods can firstly do the classification, the outcomes of which are cured in a regression-based refinement step [105], [84], [78], [166] or vice versa [75], or can do the classification and regression in a single-shot process [87], [145], [101], [106], [100], [148], [103], [102], [30], [37], [162].…”

“…DeepContext [78] is trained on the partially synthetic training depth images which exhibit a variety of different local object appearances, and real data are used to fine tune the method. The 2D-driven 3D methods and the 3D BB detectors work at the level of categories, and the 6D methods [30], [37], [166], [162] work at instance-level.…”

DeceptionNet: Network-Driven Domain Randomization

“…Another feature not well covered by synthetic data is proper illumination. Recent methods [21,29,16,56] prerender a number of synthetic images featuring different light conditions. Here, we instead implement differentiable lighting based on the simple Phong model [35], which is fully operated by the network.…”

HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects

“…When it comes to deep learning methods, training detectors on real data yields the best results. However, the fact that 3D models of the objects are available and training data can be synthesized by rendering them has been used in only a few studies, most notably using such detectors as SSD6D [20], AAE [29] and DPOD [34]. It is remarkable that all deep learning 6DoF object detectors trained either on real or synthetic data use a single neural network per object, in contrast to 2D object detectors, such as YOLO [24], SSD [21] or R-CNNs [14,13,25,15], which use one network for all object classes.…”

“…In the following sections, besides describing the steps of the dataset creation pipeline, we also present detection and 6D pose estimation results for all the newly-defined benchmarks of one of a recently introduced methods -Dense Pose Object Detector (DPOD) [34]. This method is trained on all the objects at once or on all the objects present in the test scene on strictly synthetic renderings of provided 3D models.…”