Design and evaluation of an air-to-air combat debriefing system using a head-mounted display

Abstract

The United States Air Force Red Flag exercise is the premier combat flight training experience for fighter pilots. We created and evaluated a computer system for replay of Red Flag air-to-air combat training data with alternative display systems. Air combat data could either be displayed on a console display system (CDS) which mimicked existing replay displays or in a headmounted display (HMD).The eflectiveness of replaying air combat data using these two displays was compared in a humanperformance experiment, with USAFJighter pilots as the subjects. Quantitative and qualitative data about display performance and preference were collected from the pilots who used each display to review mission replays. Although there was no statistically significant d@erence between the subject performance when using familiar CDS or the new HMD, there was a trend favoring the HMD. Red FlagRed Flag exercises held at Nellis Air Force Base NV provide some of the most challenging and beneficial training available for USAF tighter pilots. Several times a year, fighter squadrons from bases around the world participate in these exercises, where visiting combatready pilots take on the role of the Blue Force (US and allies) and fly against resident Red Force trained to mimic enemy tactics.An instrumented range north of Nellis AFB, NV provides detailed information about each aircraft in a Red Flag exercise. The Red Flag Measurement and Debriefing System (RFMDS) is collective name for the aircraft tracking and recording network and its debrief system for training feedback. Recorded missions can be replayed for critique to single ah-crews on a CDS or to an entire strike package's crews on large screen displays.As the air combat takes place over the desert range, RFMDS records position, speed and weapons firings data of the participating aircraft. Up to 36 aircraft can be equipped with advanced electronics pods and take on the role of the high-activity aircraft. For these aircraft, flight information is transmitted 10 times per second to telemetry recording stations. Participants simulate weapons fuing with the success calculated in real-time and the consequences radioed to participants.As flight information is transmitted to ground stations on the range and eventually relayed back to the main Red Flag building, it is displayed in real-time to safety monitors and recorded onto magnetic tape. After a oneto two-hour training mission, the aircraft recover back at Nellis. The aircrews gather in a large auditorium, and the mission commander conducts a debrief of all aspects of the mission. The mission commander relies on RFMDS to depict the actions taken by aircraft on both sides, and he can selectively review Blue and Red Force performance with resolution down to single aircraft. Evaluation of the participants' actions is often extremely frank and harsh; pilots are deadly serious about their performance.

Development of the Situation Awareness Flight Training and Simulation Evaluation (SAFTE) System: II: Final Development, Initial Test, and Documentation of the System

“…However, a more sophisticated analysis was also possible due to the fact that this approach resulted in four categories of response: "hits" were correct "yes" answers, "false alarms" were "yes" answers when the correct answer was no, "true rejections" were correct "no" answers, and "misses" were "no" answers when the correct answer was yes. This permitted analyses to be carried out using information theoretic metrics, as described by Kantowitz & Sorkin (1983) and Amburn (1994).…”

“…Although it has been pointed out that this type of assessment confounds workload with SA, it has proven valuable in some situations. Other subjective scales have been reported, utilizing information-theoretic approaches (Amburn, 1994) or other techniques.…”

“…As a further refinement of this type of measure, Eubanks and Killeen (1983) and Amburn (1994) have suggested that the probe be structured with "yes-no" responses in order to permit information-theoretic analyses. In effect, since each response reflects a "positive hit" (a correct yes answer), a "negative hit" (a correct no answer), a "positive miss" (an incorrect no answer), or "false alarm" (an incorrect yes answer), the data lend themselves to a 2-by-2 matrix.…”

“…Although it has been pointed out that this type of assessment confounds workload with SA, it has proven valuable in some situations. Other subjective scales have been reported, utilizing information-theoretic approaches (Amburn, 1994) or other techniques.A particularly well-conceptualized approach to measuring SA subjectively has been presented by Pew, Adams, and Tenney (1991). These authors first made a cogent distinction between the process of SA acquisition, and the actual awareness resulting from that process.…”

“…Such "awareness" is typically probed either with retrospective or concurrent "explicit" measures of SA (Fracker, 1991). Because there is some controversy concerning whether retrospective (after-themission) measures confound momentary and reflective SA, the concurrent measurement approach was incorporated into the SAFTE system, as described below.As a further refinement of this type of measure, Eubanks and Killeen (1983) and Amburn (1994) have suggested that the probe be structured with "yes-no" responses in order to permit information-theoretic analyses. In effect, since each response reflects a "positive hit" (a correct yes answer), a "negative hit" (a correct no answer), a "positive miss" (an incorrect no answer), or "false alarm" (an incorrect yes answer), the data lend themselves to a 2-by-2 matrix.…”