The Divergence of Objective and Subjective Situation Awareness: A Meta-Analysis

Endsley, Mica R.

doi:10.1177/1555343419874248

Cited by 69 publications

(61 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Observer ratings of SA overcome some problems with self-assessment, but still suffer from becoming a proxy for subjective performance ratings in that observers have limited information on a person’s mental representation of the situation. See also Endsley (in press).…”

Section: Introductionmentioning

confidence: 99%

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

2019

Self Cite

View full text Add to dashboard Cite

Objective: To examine evidence of sensitivity, predictiveness, and methodological concerns regarding direct, objective measures of situation awareness (SA). Background: The ability to objectively measure SA is important to the evaluation of user interfaces and displays, training programs, and automation initiatives, as well as for studies that seek to better understand SA in both individuals and teams. A number of methodological criticisms have been raised creating significant confusion in the research field. Method: A meta-analysis of 243 studies was conducted to examine evidence of sensitivity and predictiveness, and to address methodological questions regarding Situation Awareness Global Assessment Technique (SAGAT), Situation Present Assessment Technique (SPAM), and their variants. Results: SAGAT and SPAM were found to be equally predictive of performance. SPAM (64%) and real-time probes (73%) were found to have significantly lower sensitivity in comparison to SAGAT (94%). While SAGAT was found not to be overly memory reliant nor intrusive into operator performance, SPAM resulted in problems with intrusiveness in 40% of the studies examined, as well as problems with speed-accuracy tradeoffs, sampling bias, and confounds with workload. Concerns about memory reliance, the utility of these measures for assessing Team SA, and other issues are also addressed. Conclusion: SAGAT was found to be a highly sensitive, reliable, and predictive measure of SA that is useful across a wide variety of domains and experimental settings. Application: Direct, objective SA measurement provides useful and diagnostic insights for research and design in a wide variety of domains and study objectives.

show abstract

Section: Introductionmentioning

confidence: 99%

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, usability comprises effectiveness and efficiency (behavioral) as well as satisfaction (attitude) in order to get a comprehensive product evaluation. Attitudes and behavior are often separate dimensions that do not match in different various areas of research [11,119,120]. For a comprehensive evaluation of HMIs and driving automation features from a Human Factors perspective, it might be necessary to include both of these data sources.…”

Section: Discussion Of Findings and Recommendations For Future Ad Stumentioning

confidence: 99%

Where We Come from and Where We Are Going: A Systematic Review of Human Factors Research in Driving Automation

et al. 2020

View full text Add to dashboard Cite

During the last decade, research has brought forth a large amount of studies that investigated driving automation from a human factor perspective. Due to the multitude of possibilities for the study design with regard to the investigated constructs, data collection methods, and evaluated parameters, at present, the pool of findings is heterogeneous and nontransparent. This literature review applied a structured approach, where five reviewers investigated n = 161 scientific papers of relevant journals and conferences focusing on driving automation between 2010 and 2018. The aim was to present an overview of the status quo of existing methodological approaches and investigated constructs to help scientists in conducting research with established methods and advanced study setups. Results show that most studies focused on safety aspects, followed by trust and acceptance, which were mainly collected through self-report measures. Driving/Take-Over performance also marked a significant portion of the published papers; however, a wide range of different parameters were investigated by researchers. Based on our insights, we propose a set of recommendations for future studies. Amongst others, this includes validation of existing results on real roads, studying long-term effects on trust and acceptance (and of course other constructs), or triangulation of self-reported and behavioral data. We furthermore emphasize the need to establish a standardized set of parameters for recurring use cases to increase comparability. To assure a holistic contemplation of automated driving, we moreover encourage researchers to investigate other constructs that go beyond safety.

show abstract

“…SA measurement methods based on performance encounter problems with validity because performance and SA might be two different things (Nguyen et al, 2019 ). Endsley ( 2020 ) found that self-rating methods for SA measurements do not correlate with methods such as SAGAT, which aim to objectively measure SA knowledge. The author concluded that these methods measure different things, and self-rating methods should not be considered to measure the general SA.…”

Section: Situation Awareness and Its Measurementmentioning

confidence: 99%

“…Rather, they should be used to measure only the self-confidence about one's own SA. Combining low SA with high self-confidence might be especially dangerous (Endsley, 2020 ). Besides, methods based on self-rating after activity have the problem that the subject might have already forgotten details (Nguyen et al, 2019 ).…”

Section: Situation Awareness and Its Measurementmentioning

confidence: 99%

See 1 more Smart Citation

Situation Awareness Measurement in Remotely Controlled Cars

Linkov

Vanžura

2021

Front. Psychol.

View full text Add to dashboard Cite

This study reviews the current information concerning the measurement of the situation awareness (SA) of the teleoperated drivers of remotely controlled cars. The teleoperated drivers who drive these cars are in a remote location, and they control the cars through a communication interface. The objective methods with probes are beneficial in measuring SA on a closed circuit without real traffic. Questions specifically should address the information provided on the road by haptic sensations, such as the slope of the road and the vehicle's speed. Methods for measuring SA that involve probes and interruptions obviously are not suitable for use on public roads. A stable environment for the display and control of the communication interface is suitable for an eye tracker in measuring SA. These features also facilitate the use of subjective observer-rating methods. Both of these methods are suitable for driving on real roads because they are not intrusive. SA research in a real-road environment also should demonstrate how the SA of other drivers is affected by seeing a car without a driver. Given the remote character of driving, cultural differences in cognition may have a significant influence on the SA of the teleoperated driver.

show abstract

The Divergence of Objective and Subjective Situation Awareness: A Meta-Analysis

Cited by 69 publications

References 84 publications

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

Where We Come from and Where We Are Going: A Systematic Review of Human Factors Research in Driving Automation

Situation Awareness Measurement in Remotely Controlled Cars

Contact Info

Product

Resources

About