3D Visual Reconstruction as Prior Information for First Responder Localization and Visualization

Abstract

In professional use cases like police or fire brigade missions, coordinated and systematic force management is crucial for achieving operational success during intervention by the emergency personnel. A real-time situation picture enhances the coordination of the team. This situation picture includes not only an overview of the environment but also the positions, i.e., localization, of the emergency forces. The overview of the environment can be obtained either from known situation pictures like floorplans or by scanning the environment with the aid of visual sensors. The self-localization problem can be solved outdoors using the Global Navigation Satellite System (GNSS), but it is not fully solved indoors, where the GNSS signal might not be received or might be degraded. In this paper, we propose a novel combination of an inertial localization technique based on simultaneous localization and mapping (SLAM) with 3D building scans, which are used as prior information, for geo-referencing the positions, obtaining a situation picture, and finally visualizing the results with an appropriate visualization tool. We developed a new method for converting point clouds into a hexagonal prism map specifically designed for our SLAM algorithm. With this combination, we could keep the equipment for first responders as lightweight as required. We showed that the positioning led to an average accuracy of less than 1m indoors, and the final visualization including the building layout obtained by the 3D building reconstruction will be advantageous for coordinating first responder operations.

An Overall First Responder Tracking and Coordination Framework

“…The FootSLAM map might be slightly rotated because it is not georeferenced. A comprehensive performance analysis of the system, including a comparison to other techniques, is given in Kaiser et al 8 The accuracy of FootSLAM lies between 1-2 m without prior information and below 1 m with previous information, assuming map convergence. Finally, the FRs' positions are visualized with 3-D visualization (see Figure 1).…”

“…The performance will depend on the range of the UWB system and may degrade if a FR is too far away from another UWB unit. A more comprehensive description of the state of the art can be found in Kaiser et al 8 Overall, most of the systems for real-time 3-D reconstruction and self-localization are either computationally complex with high demands on battery power, or require infrastructure, which might not be available during an emergency; for instance, in a tunnel, in mines, or even in a building on fire. Note that it is usually impractical to send large amounts of data (as with visual sensors) in real time to the control center, and that privacy issues must be considered when using body cameras.…”