Predicting Perceived Situation Awareness of Low Altitude Aircraft in Terminal Airspace Using Probe Questions

Abstract

The purpose of the present study was to evaluate the effectiveness of subjective and objective probe questions in predicting situation awareness as measured by the Situation Awareness Rating Technique (SART). The data for this evaluation were taken from a previous investigation in which instrumentrated pilots flew automated ILS approaches into the Dallas-Fort Worth (DFW) Airport while monitoring the status of patrol vehicles proximal to their approach path. At three points during a simulation run, pilots were administered a questionnaire containing seven questions designed to probe situation awareness. At the end of the run, SART was administered. We found that certain probe questions can predict SART scores. However, the usefulness of these probes requires that the questions be designed in conjunction with scenario development to ensure that operationally critical variables are being probed, and that sufficient variability in the responses allow assessments of relations with sufficient statistical power.

Assessing the Situation Awareness of Pilots Engaged in Self Spacing

“…In SPAM scenarios, subjective assessment latencies were significantly correlated with IAS variability, similar to the findings of Strybel et al (2007). Note, however, that both SAGAT and SPAM SA measures were not related to SART ratings of SA, contrary to Strybel et al (2007).…”

“…Note, however, that both SAGAT and SPAM SA measures were not related to SART ratings of SA, contrary to Strybel et al (2007). This may be due to differences in the length of the scenarios.…”

“…The simulated environment employed in this experiment was the northwest arrival corridor to the Dallas Fort-Worth (DFW) airport, similar to that used in Strybel et al (2007). All scenarios were developed in consultation with an Air Transport Pilot and former Southern California TRACON Controller.…”

“…In a previous experiment (Strybel, et al, 2007) we showed that online probe questions were related to subjective SA as measured by a standardized SA rating instrument (Situation Awareness Rating Technique, or SART). Probe questions were administered during a simulation predicted SART scores administered at the end of a scenario.…”

The Divergence of Objective and Subjective Situation Awareness: A Meta-Analysis

“…SART scores were negatively correlated with real-time probe reaction time (Pearson’s r = −.71; Jones & Endsley, 2004), showing faster responses when SA was rated higher. SART was positively correlated with real-time probe accuracy for the SART combined score (Pearson’s r = .30) and for the understanding (Pearson’s r = .42) and demand on resource component sub-scores (Pearson’s r = .55; Strybel et al, 2007). In that real-time probes constitute a secondary task, it is possible that it is tapping into workload, which SART also assesses through its supply and demand components.…”