Meaningful Test and Evaluation of Indoor Localization Systems in Semi-Controlled Environments

Abstract

Despite their enormous potential, the use of indoor localization systems (ILS) remains seldom. One reason is the lack of market transparency and stakeholders’ trust in the systems’ performance as a consequence of insufficient use of test and evaluation (T&E) methodologies. The heterogeneous nature of ILS, their influences, and their applications pose various challenges for the design of a methodology that provides meaningful results. Methodologies for building-wide testing exist, but their use is mostly limited to associated indoor localization competitions. In this work, the T&E 4iLoc Framework is proposed—a methodology for T&E of indoor localization systems in semi-controlled environments based on a system-level and black-box approach. In contrast to building-wide testing, T&E in semi-controlled environments, such as test halls, is characterized by lower costs, higher reproducibility, and better comparability of the results. The limitation of low transferability to real-world applications is addressed by an application-driven design approach. The empirical validation of the T&E 4iLoc Framework, based on the examination of a contour-based light detection and ranging (LiDAR) ILS, an ultra wideband ILS, and a camera-based ILS for the application of automated guided vehicles in warehouse operation, demonstrates the benefits of T&E with the T&E 4iLoc Framework.

Advances in Indoor Positioning and Indoor Navigation

“…Schyga et al [ 15 ] also deal with the evaluation problem and the difficulty to compare results among different sources. They propose the T&E 4iLoc Framework, a methodology for T&E of indoor localization systems in semi-controlled environments based on a system-level and black-box approach, which is highly reproducible and, therefore, facilitates comparison.…”

Multi-Sensor Data Fusion Solutions for Blind and Visually Impaired: Research and Commercial Navigation Applications for Indoor and Outdoor Spaces

“…The scientific community has proposed and demonstrated numerous solutions that span several different techniques and approaches to address these issues. However, few of them can be used immediately by the BVI individuals to improve their life as they are largely exploratory and most commonly have little to no consideration for the utility aspect of day-to-day application [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 ]. Furthermore, contrary to outdoor space navigation, which leverages the GPS infrastructure, indoor space navigation has no adequate solution.…”

Challenges in Acceptance of Smartphone-Based Assistive Technologies: Extending the UTAUT Model for People With Visual Impairments

“…Over the last decades, solutions have been proposed for both indoor and outdoor navigation. Specifically, indoor navigation solutions are based on inertial odometry (Ren et al, 2021), sensor-based pedestrian dead reckoning (Huang et al, 2019), indoor localization utilizing computer vision and deep learning on camera-based input or beacons readings (Koutris et al, 2022;Viset et al, 2022), as well as methods for reliably evaluating the adaptability of these solutions (Schyga et al, 2022). Likewise, outdoor navigation employs approaches incorporating both the smartphone sensors and external higher accuracy GPS receivers coupled with patent-pending novel routing algorithms (Theodorou et al, 2022a); deep learning computing vision for detecting user path obstacles, car directionality, and crosswalks near traffic lights (Chandna & Singhal, 2022;Das et al, 2021;Hsieh et al, 2021;Shelton & Ogunfunmi, 2020); and "smart" traffic lights devices for the safe passage of crossings (Theodorou et al, 2022a).…”