Efficient 3D Instance Segmentation for Archaeological Sites Using 2D Object Detection and Tracking

Abstract

This paper introduces an efficient method for 3D instance segmentation based on 2D object detection, applied to the photogrammetric survey images of archaeological sites. The method capitalizes on the relationship between the 3D model and the set of 2D images utilized to compute it. 2D detections on the images are projected and transformed into a 3D instance segmentation, thus identifying unique objects within the scene. The primary contribution of this work is the development of a semi-automatic image annotation method, augmented by an object tracking technique that leverages the temporal continuity of image sequences. Additionally, a novel ad-hoc evaluation process has been integrated into the conventional annotation-training-testing cycle to determine the necessity of additional annotations. This process tests the consistency of the 3D objects yielded by the 2D detection. The efficacy of the proposed method has been validated on the underwater site of Xlendi in Malta, resulting in complete and accurate 3D instance segmentation. Compared to traditional methods, the object tracking approach adopted has facilitated a 90% reduction in the need for manual annotations, The approach streamlines precise 3D detection, establishing a robust foundation for comprehensive 3D instance segmentation. This enhancement enriches the 3D survey, providing profound insights and facilitating seamless exploration of the Xlendi site from an archaeological perspective.