The Validity of the SEEV Model as a Process Measure of Situation Awareness: The Example of a Simulated Endotracheal Intubation

Abstract

Objective In the context of anesthesiology, we investigated whether the salience effort expectancy value (SEEV) model fit is associated with situation awareness and perception scores. Background The distribution of visual attention is important for situation awareness—that is, understanding what is going on—in safety-critical domains. Although the SEEV model has been suggested as a process situation awareness measure, the validity of the model as a predictor of situation awareness has not been tested. Method In a medical simulation, 31 senior and 30 junior anesthesiologists wore a mobile eye tracker and induced general anesthesia into a simulated patient. When inserting a breathing tube into the mannequin’s trachea (endotracheal intubation), the scenario included several clinically relevant events for situation awareness and general events in the environment. Both were assessed using direct awareness measures. Results The overall SEEV model fit was good with no difference between junior and senior anesthesiologists. Overall, the situation awareness scores were low. As expected, the SEEV model fits showed significant positive correlations with situation awareness level 1 scores. Conclusion The SEEV model seems to be suitable as a process situation awareness measure to predict and investigate the perception of changes in the environment (situation awareness level 1). The situation awareness scores indicated that anesthesiologists seem not to perceive the environment well during endotracheal intubation. Application The SEEV model fit can be used to capture and assess situation awareness level 1. During endotracheal intubation, anesthesiologists should be supported by technology or staff to notice changes in the environment.

Attention: Theory, Principles, Models and Applications

“…Such validation is accomplished by correlating the SEEV-predicted PDTs with the % scans or percentage dwell time (PDT) on each AOI. It has been used both to predict the overall scanning behavior of users, but also, in some circumstances, to characterize the differences between novice and expert scan patterns (Grungeiger et al, 2021;Koh et al, 2011;). An important difference here is that experts should be more calibrated to the true expected value of an AOI, and less "distracted" or influenced by the two nuisance parameters of salience and effort.…”

Guiding Attention via a Cognitive Aid During a Simulated In-Hospital Cardiac Arrest Scenario: A Salience Effort Expectancy Value Model Analysis

“…Supporting this argument, empirical studies have shown that including salience (Steelman et al, 2011) or effort (Grundgeiger, Beckh, et al, 2020; Wickens et al, 2008) did not increase model fit of expert participants. Finally, a study (Grundgeiger et al, in press) indicated a positive association between EV model fit and situation awareness level 1 (perceiving events in the environment; Endsley, 1995) which further supports the idea that good EV model fit indicates an advantageous visual attention distribution. Therefore, the EV model fit also provides the opportunity to draw a conclusion about the quality of the visual attention distribution.…”

“…Compared to previous model attempts in healthcare, the current model fits are slightly worse. One explanation might be the less structured and more stressful emergency situation in the present study compared to common situations such as the uneventful induction of general anesthesia (Grundgeiger et al, in press; Grundgeiger, Wurmb, et al, 2020) or scrub nursing during caesarean section surgery (Koh et al, 2011). In the literature, staff reported such an emergency situation effect, in particular for the first minutes of CPR (Sachedina et al, 2019; Sjöberg et al, 2015).…”

“…To calculate the model fit, the predicted PDT is correlated with the empirically observed PDT that are collected using eye tracking (Wickens, 2015; Wickens et al, 2003). The model has been validated in different domains, such as aviation (model fit correlations >0.90; Steelman et al, 2011; Wickens et al, 2003; Wickens et al, 2008), driving (model fit correlations range ∼0.75–0.98; Bos et al, 2015; Cassavaugh et al, 2013; Horrey et al, 2006), scrub nursing during surgery (model fit correlations ∼0.70; Koh et al, 2011), and anesthesiology (model fit correlations rage ∼0.72 to ∼0.82; Grundgeiger et al, in press; Grundgeiger, Wurmb, et al, 2020).…”

“…First, we aimed to investigate whether the EV model could fit the eye tracking data of resuscitation team leaders. Based on previous research in team dyads (Grundgeiger et al, in press; Grundgeiger, Wurmb, et al, 2020) and larger operating room teams (Koh et al, 2011), we expected to achieve an acceptable model fit in a team of four people; however, this is not a given because a CPR situation is a stressful, loosely structured, and at times chaotic emergency (e.g., Sachedina et al, 2019; Sjöberg et al, 2015), which may induce more behavioral variability that may be difficult to capture in a model.…”