Geometry Consistency Of Augmented Reality Based On Semantics

Abstract

In augmented reality, for achieving geometric consistency in the perspective projection virtual-real, we propose a semantic consistency method to achieve the fusion between virtual and real objects with selected segmented objects in the real scene as references. The proposed framework maintains the three-dimensional structure of the scene by satisfying the global semantic map of the real scene. It takes the segmented objects in the scene as the basic unit, and executes the virtual and real fusion for ensuring the accuracy of the relative geometric position of the virtual objects. In addition, a multi-task network architecture is proposed to optimize the camera parameters based on the scene segmentation. The experiment results demonstrate the effectiveness of the proposed augmented reality geometric consistency framework, and confirm that our strategy has the capability of fusing the virtual and real geometric consistency.