Measurement of Situation Awareness in Dynamic Systems

Abstract

Methodologies for the empirical measurement of situation awareness are reviewed, including a discussion of the advantages and disadvantages of each method and the potential limitations of the measures from a theoretical and practical viewpoint. Two studies are presented that investigate questions of validity and intrusiveness regarding a query-based technique. This technique requires that a simulation of the operational tasks be momentarily interrupted in order to query operators on their situation awareness. The results of the two studies indicate that the query technique is not intrusive on normal subject behavior during the trial and does not suffer from limitations of human memory, which provides an indication of empirical validity. The results of other validity studies regarding the technique are discussed along with recommendations for its use in measuring situation awareness in varied settings.

The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task

“…As in Endsley and Kaber's (1999) study, the SAGAT was used to measure operator SA by freezing the tasks at random points in time during the experiment, blanking all visual display screens and administering queries concerning both the current and future states of the system. The SAGAT evaluated subject perception (Level 1 SA), comprehension (Level 2 SA) and projection (Level 3 SA) regarding information displayed during the tasks by comparing subject responses to queries on each level with actual situation data recorded by the PCs running the simulations (Endsley 1988(Endsley , 1995. The percentage of correct responses was then calculated for all queries.…”

“…Furthermore, the timing of each freeze was randomly determined within the periods of Manual control and automation during a trial, making prediction of the exact freeze time virtually impossible. Such freezes to collect SAGAT data (or even the possibility of such freezes) have not been found to affect subject performance (Endsley 1995(Endsley , 2000.…”

The Impact of Automation Assisted Aircraft Separation on Situation Awareness

“…A widely cited definition of SA specifies three levels of the construct in terms of 1) perception, 2) comprehension, and 3) projection [7].…”

“…Probe techniques have been identified as the most promising of the measures because they are sensitive to the operator and task environment [12] and can provide diagnostic information regarding the cause(s) of poor SA. The Situation Global Assessment Technique (SAGAT) and Situation Present Awareness Method (SPAM) are two commonly used SA probe techniques [7] [13]. SAGAT questions are administered by stopping the scenario at random intervals so that SA probes do not interfere with the current tasks.…”

AKTiveSA: A Technical Demonstrator System For Enhanced Situation Awareness

“…One of the approaches adopted by the University of Cardiff measures the recall of scenario-relevant information following periods of user interaction with the target system. A number of other measurement techniques for situation awareness have also been proposed, including SAGAT [26,27], SPAM [28] and SART [29], and we aim to examine and assess the relative suitability of these techniques in subsequent system evaluation studies.…”