Augmenting understanding of the relationship between situation awareness and confidence using calibration analysis

Abstract

The present study represents a preliminary examination of the relationship between situation awareness (SA) and confidence within a distributed information-sharing environment using the calibration methodology. The calibration methodology uses the indices of calibration, resolution and over/under-confidence to examine the relationship between the accuracy of the responses and the degree of confidence that one has in these responses, which leads to a measure of an operator's meta-SA. The results of this study revealed that, although the participants were slightly overconfident in their responses, overall they demonstrated good meta-SA. That is, the participants' subjective probability judgements corresponded to their pattern of SA response accuracy. It is concluded that the use of calibration analysis represents a better methodology for expanding our understanding of the relationship between SA and confidence and ultimately how this relationship can impact decision-making and performance in applied settings than can be achieved by examining SA measures alone.

The Divergence of Objective and Subjective Situation Awareness: A Meta-Analysis

“…At least one measure of SA, QUASA, has attempted to combine both an objective assessment of SA (via SAGAT like probes), along with a subjective rating of how confident the person is in their answers (Edgar, Edgar, & Curry, 2003;McGuinness, 2004). Related to the issue of meta-awareness, the appropriate calibration of a person's confidence level (e.g., degree of underconfidence or overconfidence) has also received attention (Lichacz, 2008(Lichacz, , 2009Sulistyawati, Wickens, & Chui, 2009, 2011.…”

“…People with low SA were more likely to be overconfident than those with high SA. Lichacz (2008) and Rousseau et al (2010) attribute overconfidence in SA to people's level of experience with similar, but unrelated tasks (i.e., general overconfidence in the performance of the task itself). (See also work by Einhorn &Hogarth, 1981 andKruger &Dunning, 1999 on overconfidence.…”

Prediction in Situation Awareness: Confidence Bias and Underlying Cognitive Abilities

“…4. Overconfidence would disrupt SA and, via this, would disrupt performance on a task that depends, in part, on that SA(Lichacz, 2008;Lichacz et al, 2003).We note that data regarding this study represent performance and SA from single-pilot trials; these trials were one component of an experiment that also examined shared SA(Salmon et al, 2008) between two pilots on subsequent trials. Downloaded by [University of Tennessee At Martin] at 07:31 04 October 2014…”

“…The critical mediating variable in this case is the degree of confidence one has in one's own SA. If Downloaded by [University of Tennessee At Martin] at 07:31 04 October 2014 the confidence is calibrated, then SA should be effective (Lichacz, 2008;Lichacz et al, 2003). However, if there is overconfidence, then it is likely that the worker could initiate a premature action, rather than delaying the action to seek more evidence to improve the level of SA (Alfredson, 2007).…”

Level of Automation Effects on Situation Awareness and Functional Specificity in Automation Reliance

“…Lee refers to this as low meta-SA, alluding to metacognitive regulation of situation awareness. Similarly, Lichacz attributed a weak correlation between confidence and accuracy to low meta-SA in an information gathering and conflict resolution exercise (Lichacz, 2008). Rousseau et al built on the meta-SA concept by suggesting that subjective measures of SA are information-based judgements, rather than experience based (Rousseau, Tremblay, Banbury, Breton, & Guitouni, 2010).…”