Object spatial localization by fusing 3D point clouds and instance segmentation

Abstract

Real-time detection and acquisition of localization information of instance targets in real three-dimensional space plays an important role in application scenarios such as virtual reality simulation and digital twinning. The existing spatial localization methods without the aid of lidar and other equipment often have problems in restoring the real scale. In order to overcome this problem and achieve more accurate object spatial localization, an object spatial localization by fusing 3D point clouds and instance segmentation is proposed. This method obtains sparse 3D point cloud data by binocular stereo matching, which is used to describe the real scale and spatial location information of the object. Then uses deep learning method to perform monocular instance segmentation on the specific category target of interest, and the segmentation result is used as the front/background prior information to complete the coordinate correction and densification of the 3D point cloud data inside and outside the object contour. Compared with the unsupervised depth estimation methods based on deep learning, our method can quickly and accurately achieve the three-dimensional precise localization of the instance target and its various components in real-world scenes, and the accuracy in the indoor scene is more than 90%.

Visualization of 3D cable between utility poles obtained from laser scanning point clouds: a case study

“…Therefore, VLS is a very effective tool for inspection. A technique to convert camera images into a 3D point cloud was proposed [9,10], but the accuracy of this method is inferior to using a laser point cloud depending on the photographing environment such as backlighting.…”