A human factors perspective on the use of automated aids in the evaluation of NDT data

Abstract

Abstract. In comparison to manual NDT methods, mechanized NDT is considered to be more reliable for a number of reasons, one of which being that the role of the inspectors and, therewith, the potential for human error, have been reduced. However, human-automation interaction research suggests that in spite of its numerous benefits, automation can lead to new yet unknown risks. One of those risks is inappropriate reliance on automation, which can result in automation misuse and disuse. The aim of this study was to investigate the potential inappropriate use of automation (specificallythe automated aids) in NDT addressing therewith the prevalent belief in the high reliability of automation held by the NDT community. To address this issue, 70 NDT trainees were asked to control the results of an eddy current data evaluation, allegedly provided by an automated aid, i.e. indication detection and sizing software. Seven errors were implemented into the task and it was measured to what extent the participants agreed with the aid. The results revealed signs of both misuse (agreeing with the aid even though it is incorrect) and disuse (disagreeing with the aid even though it is correct) of the aid that can affect the reliability with which inspections are carried out. Whereas misuse could be explained by a lower propensity to take risks and by a decreased verification behavior-possibly due to bias towards automation and complacent behavior-, disuse was assigned to problems in establishing the sizing criterion or to general difficulties in sizing. The implications of these results for the NDT praxis including suggestions for the decrease of automation bias are discussed.

Automated sizing and classification of defects in CANDU pressure tubes

“…Reflecting on Bertovic's (2016) paper in which it was stated that automation poses a potential risk of error within non-destructive testing, the error is largely mitigated with the current operational procedure of two analysis streams and the fact that while the procedure that an analyst follows is now automated, the procedure itself is remains auditable. Furthermore, the industry application of this research is decision support.…”

Aviation Automation and CNS/ATM-related Human-Technology Interface: ATSEP Competency Considerations

“…It has been posited that an evaluation of the consequences for human operator performance is an important consideration in taking a decision regarding the type and level of automation in any system design (Parasuraman et al, 2000). It has also been argued that because automated systems and aids may fail occasionally, people are necessarily involved as a measure of error recovery and error correction (Bertovic, 2016), thus underscoring not only the human operator as the "common unchanged element in the exponential growth of the automated systems" (Brown, 2016, p. 31) but also the imperativeness of "human-centered automation", an approach which, according to the International Civil Aviation Organization, takes "into account the human element during the design phase so that the resulting system capitalizes upon the relative strengths of humans and computer-based technology" (ICAO, 1994, p. 1).…”

“…Quite a staggering number of literature (e.g. Bainbridge, 1983;Bertovic, 2016;Billings, 1996Billings, /1995Brown, 2016;Parasuraman & Riley, 1997;Parasuraman et al, 2000;Sarter et al 1997;Woods, 2001;) has identified and addressed the risks, concerns, and challenges arising from human-automation interactions and the development of automated aids such as a range of technical issues relating to the automation of particular functions and the characteristics of associated sensors, controls, and software (Parasuraman et al, 2000), inappropriate reliance on automation, complacency, and automation bias, which, to say the least, are as relevant to the roles of the air traffic safety electronics personnel (ATSEP) as they are to the activities of air traffic controllers and airlines' flight crew. This is more so given the considerable increases in the complexity and level of automation, which create problems of situational awareness.…”

“…The risks of inappropriate reliance on automation result from incompatibility between the level of trust in automation and the actual capabilities and limitations of the automated aid. Some of the factors in the development of trust in and reliance on automation include automation reliability (Parasuraman & Riley, 1997), experience with the automated aid, and individual differences in reliance on automation (Bertovic, 2016). The National Research Council (1998) identified seven attributes of trust in automation, namely: reliability, robustness, familiarity, understandability, explication of intention, usefulness, and dependence.…”

“…It has been argued that when accurate and used correctly, relying on automated aids is a very efficient cognitive strategy leading to an accurate and reliable performance of the system while relying on less than perfect aids can lead to errors (Bertovic, 2016). Parasuraman and Riley (1997) also note that operators may not use a reliable automated system if they believe it to be untrustworthy, and they may also continue to rely on automation even when it malfunctions.…”