The ‘problem ’ with automation: inappropriate feedback and interaction, not ‘over-automation’

Abstract

As automation increasingly takes its place in industry, especially high risk industry, it is often blamed for causing harm and increasing the chance of human error when failures do occur. I propose that the problem is not the presence of automation, but rather its inappropriate design. The problem is that the operations under normal operating conditions are performed appropriately, but there is inadequate feedback and interaction with the humans who must control the overall conduct of the task. When the situations exceed the capabilities of the automatic equipment, then the inadequate feedback leads to difficulties for the human controllers. The problem, I suggest, is that the automation is at an intermediate level of intelligence, powerful enough to take over control that used to be done by people, but not powerful enough to handle all abnormalities. Moreover, its level of intelligence is insufficient to provide the continual, appropriate feedback that occurs naturally among human operators. This is the source of the current difficulties. To solve this problem, the automation should either be made less intelligent or more so, but the current level is quite inappropriate. The overall message is that it is possible to reduce error through appropriate design considerations. Appropriate design should assume the existence of error, it should continually provide feedback, it should continually interact with operators in an effective manner, and it should allow for the worst situations possible. What is needed is a soft, compliant technology, not a rigid, formal one.

Behavioural changes in drivers experiencing highly-automated vehicle control in varying traffic conditions

“…The reduced workload from the automation of driving can result in an underload that is equally hazardous to road safety as overload Parasuraman 1992, Hancock andVerwey 1997). To combat such issues, vehicle manufacturers have been urged to design automated systems that provide up-to-theminute communication about their operation (Norman, 1990). Empirical evidence 6 supports such a viewpoint, for example that prominent and understandable feedback results in a more expeditious response to ACC failure (Seppelt and Lee, 2007).…”

Malleable Attentional Resources Theory: A New Explanation for the Effects of Mental Underload on Performance

“…This solution would also satisfy the driver's need for feedback, a classic problem with automated systems (Norman, 1990).…”

The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task

“…This study points to the fact that many issues, other than the LOA itself, can affect performance. Careful attention to the design of the system and the feedback provided to the operator are required under any LOA (see Norman 1989).…”