Efficient 3D Mapping and Modelling of Indoor Scenes with the Microsoft HoloLens: A Survey

Abstract

The Microsoft HoloLens is a head-worn mobile augmented reality device. It allows a real-time 3D mapping of its direct environment and a self-localisation within the acquired 3D data. Both aspects are essential for robustly augmenting the local environment around the user with virtual contents and for the robust interaction of the user with virtual objects. Although not primarily designed as an indoor mapping device, the Microsoft HoloLens has a high potential for an efficient and comfortable mapping of both room-scale and building-scale indoor environments. In this paper, we provide a survey on the capabilities of the Microsoft HoloLens (Version 1) for the efficient 3D mapping and modelling of indoor scenes. More specifically, we focus on its capabilities regarding the localisation (in terms of pose estimation) within indoor environments and the spatial mapping of indoor environments. While the Microsoft HoloLens can certainly not compete in providing highly accurate 3D data like laser scanners, we demonstrate that the acquired data provides sufficient accuracy for a subsequent standard rule-based reconstruction of a semantically enriched and topologically correct model of an indoor scene from the acquired data. Furthermore, we provide a discussion with respect to the robustness of standard handcrafted geometric features extracted from data acquired with the Microsoft HoloLens and typically used for a subsequent learning-based semantic segmentation.

A Real-Time Distance Measurement System for a Digital Twin Using Mixed Reality Goggles

“…An interesting solution in room mapping is presented in [ 31 , 32 ]. The authors of these papers present the possibility of obtaining a digital representation of large indoor spaces in buildings using Hololens goggles.…”

Indoor Mapping with Entertainment Devices: Evaluating the Impact of Different Mapping Strategies for Microsoft HoloLens 2 and Apple iPhone 14 Pro

“…Thus, when mapping in unfurnished or less textured indoor environments, the transition area between different rooms may be a weak point in the mapping of the HoloLens, which is prone to large errors that affect the accuracy of the overall measurement data. Weinmann et al [ 45 , 46 ] demonstrated that the data acquired by the HoloLens is accurate enough to be used to reconstruct semantically rich and topologically correct indoor scene models. Jäger et al [ 47 ] in their study performed neural radiance fields (NeRFs) [ 48 , 49 , 50 ] 3D reconstruction using internal camera poses and images provided by the HoloLens, and revealed the potential of combining the HoloLens with NeRFs for highly detailed, colored, mobile 3D mapping.…”

Evaluation of Intel Realsense D455 Camera Depth Estimation for Indoor Slam Applications

“…Low-cost 3D sensors such as RGBD cameras together with SLAM-based tracking provide the means for efficient and affordable mobile mapping systems with potential applications in fields like robotics and teleoperation (Stotko et al, 2019), documentation of cultural heritage (Teruggi and Fassi, 2022), forest monitoring as well as the mapping (Hou et al, 2023) and automatic modeling (Weinmann et al, 2021; of indoor building environments.…”