Assessment of Situation Awareness Conflict Risk between Human and AI in Process System Operation

Abstract

The conflict between human and artificial intelligence is a critical issue, which has recently been introduced in Process System Engineering, capturing the observation and action conflicts. Interpretation conflict is another source of potential conflict that can cause serious concern for process safety as it is often perceived as confusion, surprise, or a mistake. It is intangible and associated with situation awareness. However, interpretation conflict has not been studied with the required emphasis. The current work proposes a novel methodology to quantify interpretation conflict probability and risk. The methodology is demonstrated, tested, and validated on a two-phase separator. The results show that interpretation conflict is usually hidden, mixed, or covered by traditional faults, and noises in observation and interpretation, including sensor faults, logic errors, cyberattacks, human mistakes, and misunderstandings, may easily trigger interpretation conflict. The proposed methodology will serve as a mechanism to develop strategies to manage interpretation conflict.

Process safety 4.0: Artificial intelligence or intelligence augmentation for safer process operation?

“…It can be difficult to obtain high-quality data in complex process plants and process a large amount of data within a reasonable time. 18,19 Additionally, training AI models with high accuracy can be critical when dealing with significant and correlated industrial data. 20 Therefore, while developing a F I G U R E 2 AI and IA-based process safety system.…”

“…In this approach, the AI assists humans, such as recommendations, predictions, and task automation, to make decisions. 18 However, humans are responsible for making the final decisions based on AI's suggestions. This approach ensures that humans maintain control and oversight over the AI system while benefiting from AI capabilities.…”

“…Implementing AI‐based automated prevention layers is challenging. It can be difficult to obtain high‐quality data in complex process plants and process a large amount of data within a reasonable time 18,19 . Additionally, training AI models with high accuracy can be critical when dealing with significant and correlated industrial data 20 .…”

“…AIiL creates a feedback loop between the manual system and the AI. In this approach, the AI assists humans, such as recommendations, predictions, and task automation, to make decisions 18 . However, humans are responsible for making the final decisions based on AI's suggestions.…”