Augment-Me: An Approach for Enhancing Pilot’s Helmet-Mounted Display Visualization for Tactical Combat Effectiveness and Survivability

Abstract

A learning framework for combining state-of-the-art augmented reality (AR) technologies and artificial intelligence (AI) for helmet-mounted display applications in combat aviation has been proposed to explore perceptual and cognitive performance factors and their influence on mission needs. The analysis originated through examining helmet-mounted display (HMD) design features and their configurations for tactical situational awareness (SA). In accomplishing this goal, the relationship between the pilot visual search and recent advancements in AI have been gauged as a background source to unlock pilot’s uncued visual search limit. In this context, the Augment-Me framework is introduced with the ability to view and organize SA information in a predictive way. The provisioning of AI-augmented fixation maps could effectively outperform current AR-HMD capabilities, facilitating human decision while pursuing the detection and compensation of the mechanisms of human error.