Collaborative VR-Based 3D Labeling of Live-Captured Scenes by Remote Users

Wursthorn

et al. 2021

PFG

Self Cite

The Microsoft HoloLens is a head-worn mobile augmented reality device. It allows a real-time 3D mapping of its direct environment and a self-localisation within the acquired 3D data. Both aspects are essential for robustly augmenting the local environment around the user with virtual contents and for the robust interaction of the user with virtual objects. Although not primarily designed as an indoor mapping device, the Microsoft HoloLens has a high potential for an efficient and comfortable mapping of both room-scale and building-scale indoor environments. In this paper, we provide a survey on the capabilities of the Microsoft HoloLens (Version 1) for the efficient 3D mapping and modelling of indoor scenes. More specifically, we focus on its capabilities regarding the localisation (in terms of pose estimation) within indoor environments and the spatial mapping of indoor environments. While the Microsoft HoloLens can certainly not compete in providing highly accurate 3D data like laser scanners, we demonstrate that the acquired data provides sufficient accuracy for a subsequent standard rule-based reconstruction of a semantically enriched and topologically correct model of an indoor scene from the acquired data. Furthermore, we provide a discussion with respect to the robustness of standard handcrafted geometric features extracted from data acquired with the Microsoft HoloLens and typically used for a subsequent learning-based semantic segmentation.

Section: Discussionmentioning

confidence: 99%

Efficient 3D Mapping and Modelling of Indoor Scenes with the Microsoft HoloLens: A Survey

Wursthorn

et al. 2021

PFG

Self Cite

Proceedings of the 2022 European Conference on Computing in Construction

“…Saran et al (2018) created an iOS application for simultaneous scanning and user-defined bounding box annotation. Furthermore, a collaborative VR system has been developed by Zingsheim et al (2021), that enables the labeling of live-captured scenes by remote users with sparse labels. Ramirez et al (2019) convert the tedious task of annotating into a playful first person shooter game, which we adopt as a partial basis for our system.…”

Section: Related Workmentioning

confidence: 99%

SemDpray: Virtual reality as-is semantic information labeling tool for 3D spatial data

Zhao¹,

Fol²,

Jiang³

et al. 2022

Capturing the as-is status of buildings in the form of 3D spatial data (e.g., point cloud or mesh) with the use of 3D sensing technologies is becoming predominant in the Architecture, Engineering, and Construction (AEC) industry. Although the act of acquiring this data has been progressively becoming more accurate and efficient with the availability of off-the-shelf solutions in the market, the act of extracting from it as-is information has not seen similar advancements. State-of-the-art practice requires experts to manually interact with the spatial data in a laborious and time-consuming process. We propose Semantic Spray (Semspray), a Virtual Reality (VR) application that provides users with intuitive and handy tools to produce semantic information on as-is 3D spatial data (mesh) of buildings. The goal is to perform this task accurately and more efficiently by allowing users to experience, interact with, and immerse themselves in the data at different scales. Semspray is a combination of two labeling modes: user-dynamic and user-static. In the user-dynamic mode, the user is fully immersed in the 3D mesh and has the ability to walk and teleport themselves within the model; in the user-static, the user can comfortably sit on a chair and handle a small-scale version of the 3D mesh to perform the labeling, in a similar manner to hand-held painting. We evaluated SemSpray's performance with a user study of ten participants and found that the combination of the different modes is able to keep the user entertained and to limit the side-effect of VR on the sensory organs, including nausea, headache, and dizziness.

“…Since 2010, when Microsoft, in cooperation with PrimeSense, released the first Kinect, consumer RGB-D cameras have been through a democratization process, becoming very appealing to many areas of application, such as robotics [ 1 , 2 ], automotive [ 3 ], industrial [ 4 ], augmented reality (AR) [ 5 ], object detection [ 6 ], 3D reconstruction [ 7 ], and biomedical field [ 8 ]. All these applications thrived by receiving depth information in addition to color.…”

Section: Introductionmentioning

confidence: 99%

An Experimental Assessment of Depth Estimation in Transparent and Translucent Scenes for Intel RealSense D415, SR305 and L515

Curto

Araújo

2022

Sensors

RGB-D cameras have become common in many research fields since these inexpensive devices provide dense 3D information from the observed scene. Over the past few years, the RealSense™ range from Intel® has introduced new, cost-effective RGB-D sensors with different technologies, more sophisticated in both hardware and software. Models D415, SR305, and L515 are examples of successful cameras launched by Intel® RealSense™ between 2018 and 2020. These three cameras are different since they have distinct operating principles. Then, their behavior concerning depth estimation while in the presence of many error sources will also be specific. For instance, semi-transparent and scattering media are expected error sources for an RGB-D sensor. The main new contribution of this paper is a full evaluation and comparison between the three Intel RealSense cameras in scenarios with transparency and translucency. We propose an experimental setup involving an aquarium and liquids. The evaluation, based on repeatability/precision and statistical distribution of the acquired depth, allows us to compare the three cameras and conclude that Intel RealSense D415 has overall the best behavior namely in what concerns the statistical variability (also known as precision or repeatability) and also in what concerns valid measurements.