Model-Based 3D Pose Estimation of a Single RGB Image Using a Deep Viewpoint Classification Neural Network

Abstract

This paper presents a model-based approach for 3D pose estimation of a single RGB image to keep the 3D scene model up-to-date using a low-cost camera. A prelearned image model of the target scene is first reconstructed using a training RGB-D video. Next, the model is analyzed using the proposed multiple principal analysis to label the viewpoint class of each training RGB image and construct a training dataset for training a deep learning viewpoint classification neural network (DVCNN). For all training images in a viewpoint class, the DVCNN estimates their membership probabilities and defines the template of the class as the one of the highest probability. To achieve the goal of scene reconstruction in a 3D space using a camera, using the information of templates, a pose estimation algorithm follows to estimate the pose parameters and depth map of a single RGB image captured by navigating the camera to a specific viewpoint. Obviously, the pose estimation algorithm is the key to success for updating the status of the 3D scene. To compare with conventional pose estimation algorithms which use sparse features for pose estimation, our approach enhances the quality of reconstructing the 3D scene point cloud using the template-to-frame registration. Finally, we verify the ability of the established reconstruction system on publicly available benchmark datasets and compare it with the state-of-the-art pose estimation algorithms. The results indicate that our approach outperforms the compared methods in terms of the accuracy of pose estimation.

Voxel-Based Scene Representation for Camera Pose Estimation of a Single RGB Image

“…In most studies, a sparse 3D model consisting of an image set of scenes and 3D points (sparse 3D model) has been used for deep learning-based camera pose estimation [12][13][14][15]19]. Specifically, 2D-3D matching has been investigated in coarse-to-fine localization studies focusing on improving the performance of deep learning methods [14,15].…”

“…The target of these studies is mainly the robotic manipulation of specific objects, such as chairs and boxes, and human-robot interaction. Deep neural networks have also been used for 6-DoF tracking of rigid objects in large occluded environments [16,19]. However, because these methods require red, green, blue, and depth (RGB-D) sensing data, their practical use is constrained to indoor scenes.…”

Fixed-Wing Unmanned Aerial Vehicle 3D-Model-Based Tracking for Autonomous Landing

“…Some methods, as described for pose estimation, also use RGB-D cameras to perform data fusion for tracking [58][59][60]. We can use a system designed for single- [61] or multi-object [62] tracking that can perform the tracking of unknown objects (without any a priori information) [63,64] or use the object information (e.g., size or color) [65,66] to develop our tracking algorithms. Currently, most applied pose tracking methods rely on deep learning methods to obtain better performances without having to hand-craft features that can discriminate the tracked object on the search space (image frame when using a RGB camera).…”