Constructing Category-Specific Models for Monocular Object-SLAM

Abstract

We present a new paradigm for real-time objectoriented SLAM with a monocular camera. Contrary to previous approaches, that rely on object-level models, we construct category-level models from CAD collections which are now widely available. To alleviate the need for huge amounts of labeled data, we develop a rendering pipeline that enables synthesis of large datasets from a limited amount of manually labeled data. Using data thus synthesized, we learn categorylevel models for object deformations in 3D, as well as discriminative object features in 2D. These category models are instance-independent and aid in the design of object landmark observations that can be incorporated into a generic monocular SLAM framework. Where typical object-SLAM approaches usually solve only for object and camera poses, we also estimate object shape on-the-fly, allowing for a wide range of objects from the category to be present in the scene. Moreover, since our 2D object features are learned discriminatively, the proposed object-SLAM system succeeds in several scenarios where sparse feature-based monocular SLAM fails due to insufficient features or parallax. Also, the proposed categorymodels help in object instance retrieval, useful for Augmented Reality (AR) applications. We evaluate the proposed framework on multiple challenging real-world scenes and show -to the best of our knowledge -first results of an instance-independent monocular object-SLAM system and the benefits it enjoys over feature-based SLAM methods.

The Earth Ain't Flat: Monocular Reconstruction of Vehicles on Steep and Graded Roads from a Moving Camera

“…Keypoint Network Training: The proposed network was trained on the Torch framework [25], with data comprising about 2.4 million images, generated synthetically using the modified render pipeline presented in [16]. For training and validation respectively, the generated data was split in a 75-25 ratio.…”

Feature-based visual simultaneous localization and mapping: a survey

“…Otherwise, the detection is associated with its closest landmark. In a recently proposed SLAM solution [95], objects are detected and characterized at the category level rather than just the instance level. This is based on the fact that all objects in one class have common 3D points irrespective of their different categories.…”

A Variational Observation Model of 3D Object for Probabilistic Semantic SLAM

“…For the case of landmarks, object-oriented SLAM basically deals with finite static targets [3][18] [20]; therefore, to get optimal labels l c = l = (l c , l i ) of (4), it is necessary to perform maximum likelihood estimation (MLE) by substituting the all static landmark labels that are previously known. Proposed network architecture.…”

“…Also, they rarely consider the viewpointindependent feature for 3D shape and object orientation in formulation since they adopt edge and corner filter based deformable part detection algorithm. In [20], object pose and shape optimization as well as robot trajectory are estimated in back-end using key point matching and PCA-based object observation factor. Their observation factor, however, barely considers the tractable object observation model, which is hardly adopt to complete SLAM formulation with computing observation probability.…”