Does automation bias decision-making?

Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography

“…Since the CAD tool's specificity is very low, many of the prompts it places on the mammograms can be seen as distracting and it may turn out that, in many instances, the absence of prompts is more informative than their presence. This is consistent with results of earlier human factors studies of "automation bias" (6,7), which looked at the effect of wrong computer advice on students' decisions in laboratory settings simulating aviation monitoring tasks. The kinds of behaviours we encountered in our studies are consistent with what these studies describe as errors of omission, namely, failure to take appropriate action because the automated tool fails to detect or indicate an event or irregularity.…”

“…Normal (N=30) n/a 12 (29-34) 18 (6)(7)(8)(9)(10)(11) Note. The numbers in bold type correspond to the actual numbers in the new data set.…”

Why Are People’s Decisions Sometimes Worse with Computer Support?

“…In this study, they encountered both omission and commission errors. These findings were then replicated with non-pilot samples (student participants) in laboratory settings simulating aviation monitoring tasks [13]. They found that, when the automated tool was reliable, the participants in the automated condition made more correct responses.…”

“…Skitka and colleagues [13] focused on the misuse of automation, in particular on the "automation bias" effects occurring when people used wrong computer advice for monitoring tasks in aviation. They distinguished two types of computer-induced error: a) errors of commission: decision-makers follow automated advice even in the face of more valid or reliable indicators suggesting that the automated aid is wrong; b) errors of omission: decision makers do not take appropriate action, despite non-automated indications of problems, because the automated tool did not prompt them.…”

“…Factors that have been investigated in empirical studies as possible influences in people's vulnerability to automation bias include: individual differences among operators [5,13,16,17]; people's accountability for their own decisions [17]; the levels of automation at which the computer support is provided [12,18]; the location of computer advice/warnings with respect to raw data or other non-automated sources of information [19,20]; people's exposure to automation failures [21].…”