We describe the flight testing and the integration process of the Microsoft HoloLens 2 as head-mounted display (HMD) with DLR's research helicopter. In the previous work, the HoloLens was integrated into a helicopter simulator. Now, while migrating the HoloLens into a real helicopter, the main challenge was the head tracking of the HoloLens, because it is not designed to operate on moving vehicles. Therefore, the internal head tracking is operated in a limited rotation-only mode, and resulting drift errors are compensated for with an external tracker, several of which have been tested in advance. The fusion is done with a Kalman filter, which contains a non-linear weighting. Internal tracking errors of the HoloLens caused by vehicle accelerations are mitigated with a system identification approach. For calibration, the virtual world is manually aligned using the helicopter's noseboom. The external head tracker (EHT) is largely automatically calibrated using an optimization approach and therefore, works for all trackers and regardless of its mounting positions on vehicle and head. Most of the pretests were carried out in a car, which indicates the flexibility in terms of vehicle type. The flight tests have shown that the overall quality of this HMD solution is very good. The conformal holograms are almost jitter-free, there is no latency, and errors of lower frequencies are identical with the performance that the EHT can provide, which in combination greatly improves immersion. Profiting from almost all features of the HoloLens 2 is a major advantage, especially for rapid research and development. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
In the offshore environment helicopters are widely used to transport crew and material from and to maritime wind farms. Due to unforeseeable and often inclement weather situations and challenging tasks these missions put a high workload on the helicopter pilots. In this paper two test campaigns are described which assess the utility of an affordable commercial-off-the-shelf (COTS) head-mounted display (HMD) to reduce workload for commercial maritime operations. The HMD system was implemented within the air vehicle simulator (AVES) at the German Aerospace Center (DLR). Three missions were flown with experienced offshore pilots, performed in a realistic scenario. Independent subjective assessments of both workload and situational awareness were obtained. Results from the studies show that the overall workload for all missions decreased and situational awareness increased when using the HMD. Opinions regarding overall benefit and advantages of the system were found to vary between pilots and missions.
In modern helicopters highly accurate flight state data is available through precise sensors and complex fusion algorithms in the aircraft. This data is used by the avionic systems and transported to the pilot through sophisticated human machine interfaces (HMI). Especially in helicopters, the pilot is in most flight phases directly involved in the control of the aircraft and uses the information provided by the HMI. However, in most civil aircraft no sensor systems exist to get an insight about the pilots physiological state. Even in experimental helicopters there is no real-time information about the pilots mental state and this information is typically gathered through questionnaires in the debriefing. One way to close this gap is to use physiological measurement systems such as eye tracking. In our work we integrated an eye tracking system into the experimental system of DLR's research simulator AVES and the research helicopter ACT/FHS. In this paper we describe the first steps, which include the selection of the systems, technical aspects of the hardware and software integration process and first experiments in DLR's Bo 105 helicopter and DLR's AVES simulator. Details of our developed toolchain for the live data conditioning are given and first results of combined helicopter state and eye tracking data are presented. In the end we give an outlook on the next integration steps, which include the combination with a high-fidelity head tracking system.
We describe the flight testing and the integration process of the Microsoft HoloLens 2 as head-mounted display (HMD) with DLR's research helicopter. In the previous work, the HoloLens was integrated into a helicopter simulator. Now, while migrating the HoloLens into a real helicopter, the main challenge was the head tracking of the HoloLens, because it is not designed to operate on moving vehicles. Therefore, the internal head tracking is operated in a limited rotation-only mode, and resulting drift errors are compensated for with an external tracker, several of which have been tested in advance. The fusion is done with a Kalman filter, which contains a non-linear weighting. Internal tracking errors of the HoloLens caused by vehicle accelerations are mitigated with a system identification approach. For calibration, the virtual world is manually aligned using the helicopter's noseboom. The external head tracker (EHT) is largely automatically calibrated using an optimization approach and therefore, works for all trackers and regardless of its mounting positions on vehicle and head. Most of the pretests were carried out in a car, which indicates the flexibility in terms of vehicle type. The flight tests have shown that the overall quality of this HMD solution is very good. The conformal holograms are almost jitter-free, there is no latency, and errors of lower frequencies are identical with the performance that the EHT can provide, which in combination greatly improves immersion. Profiting from almost all features of the HoloLens 2 is a major advantage, especially for rapid research and development. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.