Are the Intrusive Effects of SPAM Probes Present When Operators Differ by Skill Level and Training?

Silva, Hector I.; Ziccardi, Jason; Grigoleit, Tristan; Battiste, Vernol; Strybel, Thomas Z.; Vu, Kim‐Phuong L.

doi:10.1007/978-3-642-39209-2_31

Cited by 5 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on these studies, SPAM appears to create significant interference with performance and increases operator workload. Keeler et al (2015) implemented SPAM with computerized queries rather than verbal queries, similarly to Bacon and Strybel (2013) and Silva et al (2013). None of these 3 studies found SPAM to be intrusive.…”

Section: Intrusivenessmentioning

confidence: 99%

“…While the time to accept a probe is a secondary workload indicator, this ability to delay answering probes, or to ignore them completely, creates a bias in the measure in favor of lower workload periods. While Bacon and Strybel (2013), Silva et al (2013), and Keeler et al, (2015) reported that instructing participants to only answer SPAM queries if they felt able to and providing computerized SPAM queries rather than verbal queries could reduce SPAM intrusiveness, this method also systematically skews SA probes into low-workload periods.…”

Section: Sampling Biasmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

2019

View full text Add to dashboard Cite

Objective: To examine evidence of sensitivity, predictiveness, and methodological concerns regarding direct, objective measures of situation awareness (SA). Background: The ability to objectively measure SA is important to the evaluation of user interfaces and displays, training programs, and automation initiatives, as well as for studies that seek to better understand SA in both individuals and teams. A number of methodological criticisms have been raised creating significant confusion in the research field. Method: A meta-analysis of 243 studies was conducted to examine evidence of sensitivity and predictiveness, and to address methodological questions regarding Situation Awareness Global Assessment Technique (SAGAT), Situation Present Assessment Technique (SPAM), and their variants. Results: SAGAT and SPAM were found to be equally predictive of performance. SPAM (64%) and real-time probes (73%) were found to have significantly lower sensitivity in comparison to SAGAT (94%). While SAGAT was found not to be overly memory reliant nor intrusive into operator performance, SPAM resulted in problems with intrusiveness in 40% of the studies examined, as well as problems with speed-accuracy tradeoffs, sampling bias, and confounds with workload. Concerns about memory reliance, the utility of these measures for assessing Team SA, and other issues are also addressed. Conclusion: SAGAT was found to be a highly sensitive, reliable, and predictive measure of SA that is useful across a wide variety of domains and experimental settings. Application: Direct, objective SA measurement provides useful and diagnostic insights for research and design in a wide variety of domains and study objectives.

show abstract

Section: Intrusivenessmentioning

confidence: 99%

Section: Sampling Biasmentioning

confidence: 99%

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

2019

View full text Add to dashboard Cite

show abstract

“…For example, Pierce (2012) found that SPAM negatively affected performance of novices on an ATCo task, as well as increasing mental workload to a level similar to other conditions where participants had to do the ATCo tasks while carrying out cognitively demanding secondary tasks. On the other hand, Silva et al (2013) found no evidence that SPAM probes were intrusive to performance or affected student ATCos differently. Nevertheless, given the suggestion of probe interference, it is therefore best to use SPAM when workload is moderate.…”

Section: Implications For Sa Measurementmentioning

confidence: 76%

A Situated Approach to the Understanding of Dynamic Situations

Chiappe

Strybel

2015

Journal of Cognitive Engineering and Decision Making

Self Cite

View full text Add to dashboard Cite

Situation awareness (SA) is an operator's understanding of "what is going on" as he or she manages a complex, dynamic system. In what follows, the case for a situated approach to SA is presented. It claims operators rely on interactions between internal and external representations to maintain their understanding of situations. It is argued that this approach allows researchers to capture two different types of SA-one where operators store SA information directly and one where the information is offloaded but the operator internally represents where to access it from the world and can do so quickly. This position takes a broader unit of analysis on SA than the widely cited Three-Level Model, which claims that only if information is stored internally does it count as SA. We also discuss how to assess SA in situated agents, focusing on the Situation Present Assessment Method (SPAM). We end by discussing some of the potentially fruitful research questions that arise as a result of taking a situated approach as well as implications for interface design.

show abstract

“…Such parallel activity may be the cause of the decline in the performance of most respondents. Although participants were given the exibility to determine when they were ready, answering SA questions with the main task causes secondary tasks that interfere with the performance of the main task (Silva et al 2013).…”

Section: Predictive Validitymentioning

confidence: 99%

Situation Awareness Measurement in Pedestrian Environments: A Comparison between Freeze-probing and Realtime-probing technique

Akbar

Wijayanto

Hartono

2022

Preprint

View full text Add to dashboard Cite

Pedestrian safety is affected by their environment. Reduced pedestrian awareness is generally due to walking being an activity repeatedly every day. Situation Awareness (SA) measurements can evaluate a pedestrian's ability to handle environmental situations. This refers to the importance of knowing the SA of pedestrians using effective measurement methods. This study aims to evaluate methods of measuring SA based on the effectiveness of its measure in a pedestrian environment by comparing the freeze-probing and real-time-probing techniques. Participants were eleven pedestrians (seven male) with an average age of 22 (SD ± 1.3) years old and walking habits for 15 (SD ± 8.5) minutes/day. Participants watched a video of a pedestrian walking on a treadmill with three speed levels. SA was measured by freeze and realtime-probing in different scenarios. Performance was determined based on verbal protocol. The realtime-probing has better sensitivity, but it has more problems with intrusiveness. Based on the regression results between predictors in the form of SA scores and performance as dependent variables, this probe shows better predictive results at low workload conditions, which corresponds to the characteristics of the pedestrian domain. Based on its advantages, real-time-probing is considered more effective for SA measurement needs in pedestrian environments that tend to be simple, stable, and do not cause high workloads. An effective measurement method will result in an accurate measure of SA score that can be the basis for developing the concept of SA and designing new designs or procedures in the pedestrian environment.

show abstract

Are the Intrusive Effects of SPAM Probes Present When Operators Differ by Skill Level and Training?

Cited by 5 publications

References 9 publications

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

A Situated Approach to the Understanding of Dynamic Situations

Situation Awareness Measurement in Pedestrian Environments: A Comparison between Freeze-probing and Realtime-probing technique

Contact Info

Product

Resources

About