Measuring the Impact of Console Operator Experience in a Simulated Petrochemical Refining Emergency Event

Abstract

Critical process industries such as petrochemical refining have actively sought to make their operations safer and more efficient. In doing this, the industry has found success in automating systems. However, increasing levels of automation is known to have some negative effects on the human operator (Kaber & Endsley, 1997). Consequently, operators have had less opportunity to be exposed to and engage in managing emergency events due to reliable automation. The current investigation explores the role that console operator experience has in the management of emergency events and in the maintenance of situation awareness within the petrochemical industry. Incumbent Console Operators completed several trials of a simulated emergency event where prior exposure to the live event, performance, workload, and situation awareness metrics were collected. The results suggest that experience with managing the live event had little effect on the collected metrics. The potential implications of these results are discussed.

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

“…A number of studies have attempted to directly compare SAGAT and SPAM, focusing on their relative abilities to predict performance in the simulation, with differing conclusions (Durso, Bleckley, & Dattel, 2006;Durso et al, 1998;Jones & Endsley, 2004;Loft, Bowden, et al, 2015;Strybel, Vu, Kraft, & Minakata, 2008). In addition, many researchers have sought to determine how sensitive these techniques are to the independent manipulation provided in the experiment (Alexander & Wickens, 2005;Jones & Endsley, 2004;Silva, Grigoleit, Ann Burress, & Fitzpatrick, 2017;Vidulich, 2000). A goal of the present study is to examine the existing research base to address these questions and to determine how well these two approaches to objective SA measurement fare in terms of sensitivity and predictive ability.…”

“…Six studies included a direct comparison of the sensitivity of SAGAT and either SPAM or real-time queries in the same study. Two studies found SAGAT more sensitive than real-time probes (Endsley, Sollenberger, & Stein, 2000;Jones & Endsley, 2004), three studies found SPAM more sensitive than SAGAT (Alexander & Wickens, 2005;Cummings & Guerlain, 2007;Silva et al, 2017), and one study found mixed results with SAGAT and real-time probes equally sensitive to the study manipulations (Burns et al, 2008).…”

“…Of the three studies where SPAM was more sensitive than SAGAT, one measured SAGAT only at the end of two trials, which created insufficient data (Cummings & Guerlain, 2007). In Silva et al (2017), while SAGAT was not sensitive to operator experience level, neither were the performance measures captured. Experienced operators had better accuracy on SPAM however.…”

Situation Awareness Measurement in Pedestrian Environments: A Comparison between Freeze-probing and Realtime-probing technique

“…Another study using SAGAT and SPAM in an emergency event said that neither SPAM nor SAGAT could distinguish the effects of 'workload' treatment. However, SPAM accuracy is more sensitive in capturing differences in SA scores based on the effect of participants 'experience' (Silva et al 2017).…”