Structure from Motion with Objects

Camera Relocalization with Ellipsoidal Abstraction of Objects

“…To simulate this behaviour, we first project the groundtruth 3D object point cloud into the image using the groundtruth camera projection matrix, and then compute the bounding box of obtained 2D points. The ellipse that inscribes the bounding box is finally used as an approximation of the projected object, as suggested in [3,28] We randomly pick 50 frames per object (the dataset contains 15 objects, with roughly 1200 images per each) to build their ellipsoidal models using [28]. All the other frames are used for testing.…”

“…Modeling object projections by virtual ellipses allowed Crocco et al to propose a closed-form solution for SFM reconstruction of the scene in the form of an ellipsoid cloud [3]. However, this method is limited to the case of an orthographic projection, as well as its extension integrating CAD object models for higher reconstruction accuracy [6].…”

Camera Pose Estimation with Semantic 3D Model

“…We first evaluated the performance of our cost function minimization on a generated 3D environment, this in comparison with two other errors from the state of the art. More precisely, the QuadricSLAM method [16] iteratively minimizes a geometric error defined as the distance between the bounding boxes of detected and reprojected ellipses, depending on the six pose parameters, while the analytical solutions presented in [14], [19] are based on an algebraic distance between the vectors formed by the 5 parameters characterizing ellipses in homogeneous coordinates. Note that a closed form solution for pose can be computed when a large number (≥ 12) of correspondences are available [25].…”

“…For an in-depth evaluation of our method, we have tested it on the publicly available TUM RGB-D Dataset [26] (sequence Fr2/Desk). Objects were first detected using YOLOv3 [13], then virtual ellipses were fitted to the bounding boxes, as suggested in [14], [19]. The model was then built using [19].…”

“…In particular, extraction and matching of high-level features (objects), instead of the traditional low-level primitives (visual keypoints, line segments), is already at the basis of several methods in the literature. Modeling object projections by virtual ellipses allowed Crocco et al to propose a closed-form solution for SfM reconstruction of the scene in the form of an ellipsoid cloud [14]. However, this method is limited to the case of an orthographic projection, as well as its extension integrating CAD object models for higher reconstruction accuracy [15].…”