Camera Pose Estimation with Semantic 3D Model

Abstract

In computer vision, estimating camera pose from correspondences between 3D geometric entities and their projections into the image is a widely investigated problem. Although most state-of-the-art methods exploit simple primitives such as points or lines, and thus require dense scene models, the emergence of very effective CNN-based object detectors in the recent years have paved the way to the use of much lighter 3D models composed solely of a few semantically relevant features. In that context, we propose a novel model-based camera pose estimation method in which the scene is modeled by a set of virtual ellipsoids. We show that 6-DoF camera pose can be determined by optimizing only the three orientation parameters, and that at least two correspondences between 3D ellipsoids and their 2D projections are necessary in practice. We validate the approach on both simulated and real environments.

Perspective-2-Ellipsoid: Bridging the Gap Between Object Detections and 6-DoF Camera Pose

“…(ii) The position that best satisfies the ellipse -ellipsoid correspondences given each camera orientation is then computed based on the theoretical considerations presented in [18], [19].…”

“…In this section, we present the process of camera pose estimation in the minimal case of two 2D-3D correspondences. The method exploits the inherent decoupling between camera orientation and position arising from the ellipse-ellipsoid modeling paradigm, which was introduced in [18], [19], and derives an approximated analytical expression of the complete camera pose as a function of only one angular parameter.…”

“…That solution proposes to minimize a geometric reprojection error as a function of the camera's six DoF, based on initial position and orientation values provided by odometric sensors. Recently, it has been shown that the problem of camera pose estimation from ellipse-ellipsoid correspondences has at most 3 DoF, since the camera position can be inferred from its orientation, provided that at least one ellipseellipsoid correspondence is known [18], [19]. In particular, the possibility to compute a rough estimate of the pose from the camera orientation acquired by sensors or computed from vanishing points was demonstrated in [19].…”

“…In particular, the possibility to compute a rough estimate of the pose from the camera orientation acquired by sensors or computed from vanishing points was demonstrated in [19]. Recovering the full camera pose from at least two objects was investigated on synthetic data in [18]. However, a prior on orientation was required and the method has proven sensitive to noise on ellipses as well as to the number of ellipses detected in the image.…”

“…Given two detected objects, the method presented in section II allows the camera orientation to be recovered as a function of only one angular parameter under two assumptions satisfied by many robotics applications. Given the possibility to derive position from orientation [18], the camera pose is the one that minimizes the ellipse-ellipsoid reprojection error. A robust method is then presented in section III to handle any number of objects.…”