Optimal 6D Object Pose Estimation with Commodity Depth Sensors

Abstract

Accurate 6D object pose estimation, as well as other model-based shape matching objectives such as object detection, classification, and shape inspection, is prominent and necessary in a large number of domains and applications. This includes household and robotics applications to automate various tasks, where input depth measurements are aligned to a corresponding model of the object, such as a CAD that is composed of several interconnected parts, depending on the model's level of detail and complexity. Industrial metrology systems also compare the shapes of engineered components and equipment to their corresponding 3D models for quality assessment and anomaly detection; this often requires very costly and high resolution projection and detection mechanisms. Structuredlight laser scanners have, however, emerged as a viable option to provide cheap and reasonably accurate depth estimates, which have recently become commodity-priced within this decade due to the maturity of the technology. These sensors are also robust for semi-transparent and reflective surfaces, dynamic scenery, ambient background light, and temperature drift. Certain structured-light light coding methods unfortunately need to implement a complicated, non-invertible transformation of a distorted light pattern to generate the pixel-by-pixel depth estimates, thereby leading to a loss of information. This can result in poorly estimated or even missing depth data, especially when the object has small 3D features or non-ideal surface properties. Moreover, the informative rigid body constraint is blurred when the nonlinear 2D depth image to 3D pseudo-measurement point cloud transformation is applied, which is the domain where most model-based shape matching methods operate.Alternatively, estimating pose from information early in the structured-light sensing/processing chain has the potential to alleviate errors induced in subsequent nonlinear processing steps that are in fact locally scene dependent. This motivates one of the main contributions from this dissertation, which presents an asymptotically optimal maximum likelihood estimation method that operates directly on the raw output IR images. This is made possible by an extensive study on a commonly utilized commodity-priced structured-light sensor, which contributed to the proposed high-fidelity IR and depth image predictor and simulator that models the physics of the transmit and receive optics,

DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition

“…Figure 8(A) shows how the distance between the plane and the sensor influences the standard depth error in the resulting scans. The trend in our synthetic data matches well the one observed in experimental scans, and Choo et al model recalibrated by Landau on the same experimental data [24]. As noted in [24], these models are based on experimental results which are inherently correlated to the characteristics of their environment and sensor.…”

“…The trend in our synthetic data matches well the one observed in experimental scans, and Choo et al model recalibrated by Landau on the same experimental data [24]. As noted in [24], these models are based on experimental results which are inherently correlated to the characteristics of their environment and sensor. We could expect other data not to perfectly align with such models (as proved by the discrepancies among them).…”

“…To validate the correctness of our simulation pipeline, we first replicate the set of experiments used by Landau et al [25,24], to compare the depth error induced by Depth-Synth to experimental values, as well as to the results from Landau et al [25,24], from BlenSor [17] and from 3 Kinect error models-respectively from Menna et al [31], Nguyen et al [34] and Choo et al [9,24]. All datasets consist of scans of a flat surface placed in front of the sensor at various distances and tilt angles to the focal plane.…”

“…All datasets consist of scans of a flat surface placed in front of the sensor at various distances and tilt angles to the focal plane. The experimental data was kindly provided by Landau [25,24]. Figure 8(A) shows how the distance between the plane and the sensor influences the standard depth error in the resulting scans.…”

“…Detailed visual comparison with BlenSor[17] highlighting the salient differences, based on the noise study presented in Subsection 3.1. Detailed visual comparison with Landau's solution[25,24], based on the noise study presented in Subsection 3.1.…”