Augmented reality technology for day/night situational awareness for the dismounted Soldier

Abstract

This paper describes Applied Research Associates' (ARA) recent advances in Soldier augmented reality (AR) technology. Our AR technology, called ARC4, delivers heads-up situational awareness to the dismounted warfighter, enabling non-line-of-sight team coordination in distributed operations. ARC4 combines compact head tracking sensors with advanced pose estimation algorithms, network management software, and an intuitive AR visualization interface to overlay tactical iconic information accurately on the user's real-world view. The technology supports heads-up navigation, blue-force tracking, target handoff, image sharing, and tagging of features in the environment. It integrates seamlessly with established network protocols (e.g., Cursor-on-Target) and Command and Control software tools (e.g., Nett Warrior, Android Tactical Assault Kit) and interfaces with a wide range of daytime see-through displays and night vision goggles to deliver real-time actionable intelligence, day or night. We describe our pose estimation framework, which fuses inertial data, magnetometer data, GPS, DTED, and digital imagery to provide measurements of the operator's precise orientation. These measurements leverage mountainous terrain horizon geometry, known landmarks, and sun position, enabling ARC4 to achieve significant improvements in accuracy compared to conventional INS/GPS solutions of similar size, weight, and power. We detail current research and development efforts toward helmet-based and handheld AR systems for operational use cases and describe extensions to immersive training applications.

Designing Critical and Secondary Information in Augmented Reality Headsets for Situational Awareness

“…Prior work has also examined employing AR headsets in the military and security domains [11,30,35,36]. In [11], the authors developed an AR headset system that displays tactical information (e.g., navigation waypoints) for soldiers on foot. The authors in [35] analyzed using an AR headset to show real-time navigational information for the US Coast Guard.…”

“…Since AR keeps users aware of their surroundings while providing additional virtual information in realtime, AR has the capability to increase users' situational awareness. Prior work has examined applying AR headsets for users' situational awareness in a wide range of contexts, such as the military [11], maintenance [12], construction [13], and healthcare [14]. However, prior work has mainly studied the applicability of AR instead of investigating how to design the visual information for aiding users' situational awareness (e.g., [14][15][16][17]).…”

Survey on Urban Warfare Augmented Reality

“…This system provides commanders real world vision with accurately registered battlefield situational information, and showed the general COP to commanders and squads. ARC4 system also can provide the function of tracking, navigation, target delivery, image sharing, and tagging of features in the environment [9]. The system is also used in military training, in which virtual battlefield environment and virtual forces are superimposed on real battlefield.…”

AR DriveSim: An Immersive Driving Simulator for Augmented Reality Head-Up Display Research

“…While once the provenance of select academic and government labs, augmented reality (AR) has now been applied in many contexts and delivered over a myriad of hardware technologies. Successes have been documented regarding, for example, smartphone AR on the go (DüNser et al, 2012 ; Shea et al, 2017 ), tablet based AR in classrooms (Bower et al, 2014 ), spatial AR in architecture (Tonn et al, 2008 ), and head-worn AR in military and medical applications (Shen et al, 2013 ; Gans et al, 2015 ). However, handheld AR notwithstanding, it is quite possible that the largest AR user base will soon be automobile drivers using see-through automotive head-up displays (HUDs) to view both screen-positioned 2D and conformal 3D AR content.…”