Measuring Situation Awareness in complex systems: Comparison of measures study

Abstract

Situation Awareness (SA) is a distinct critical commodity for teams working in complex industrial systems and its measurement is a key provision in system, procedural and training design efforts. This article describes a study that was undertaken in order to compare three different SA measures (a freeze probe recall approach, a post trial subjective rating approach and a critical incident interview technique) when used to assess participant SA during a military planning task. The results indicate that only the freeze probe recall method produced a statistically significant correlation with performance on the planning task and also that there was no significant correlation between the three methods, which suggests that they were effectively measuring different things during the trials. In conclusion, the findings, whilst raising doubts over the validity of post trial subjective rating and interviewbased approaches, offer validation evidence for the use of freeze probe recall approaches to measure SA. The findings are subsequently discussed with regard to their implications for the future measurement of SA in complex collaborative systems. Relevance to IndustrySituation Awareness is a critical commodity for teams working in industrial systems and designers and analysts alike require reliable and valid methods for assessing the impact of new systems, interfaces, training programs and procedures on the level of situation awareness held by teams. This article presents a review and comparison of situation awareness measurement approaches for use in complex industrial systems and provides recommendations on the types of methods to use during situation awareness assessments.

Hazard prediction discriminates between novice and experienced drivers

“…Salmon and Stanton, 2013). There are a variety of approaches to challenge the predominant perception, comprehension, prediction framework proposed by Endsley (1988a;1995), and researchers debate whether SA is the process of gaining awareness of the situation, or whether it is the product that reflects that awareness (see Salmon et al, 2009, for a concise review). However, as mentioned earlier, this paper is not concerned with the theoretical application of the SA framework to the field of hazard perception per se, but merely to investigate the specific role of hazard prediction using tools derived from the SA field.…”

Providing Information on the Spot: Using Augmented Reality for Situational Awareness in the Security Domain

“…performance measures (Salmon et al 2009). Very little measurement approaches exist for distributed or team situational awareness.…”

“…Self-rating techniques such as the SART questionnaire are administered post-trial, and thus have a non-intrusive character. Furthermore, in their study, (Salmon et al 2009) come to the conclusion that a post-test self-rating technique is applicable whenever "SA content is not predefined and the task is dynamic, collaborative, and changeable and the outcome is not known (e.g. real world tasks)" (Salmon et al 2009).…”

“…SA includes thus the understanding and comprehension of a given environment and situation, as context for one's own actions. In this view, SA is seen as a cognitive product of information-processing (Salmon et al 2009). The concept of SA has been used in several other domains such as energy distribution, nuclear power plant operational maintenance, process control, maritime, or tele-operations (Salmon et al 2008).…”

A situation risk awareness approach for process systems safety

“…data) in a situation wait for operators to become aware of them, operate on them, and attach meaning to them. Many other researchers Also argue that further investigation to develop the measurement of SA in complex and dynamic systems is required (Gorman et al, 2006;Salmon et al, 2009).…”