A Joint Human-Automation Cognitive System to Support Rapid Decision-Making in Hostile Environments

Abstract

Honeywell has designed a joint humancomputer cognitive system to support rapid decision making demands of dismounted soldiers. In highly networked environments the sheer magnitude of communication amid multiple tasks could overwhelm individual soldiers. Key cognitive bottlenecks constrain information flow and the performance of decision-making, especially under stress. The adaptive decision-support system mitigates non-optimal human performance via automation when the system detects a breakdown in the human's cognitive state. The human's cognitive state is assessed in real-time via a suite of neuro-physiological and physiological sensors. Adaptive mitigation strategies can include task management, optimizing information presentation via modality management, task sharing, and task loading. Mitigations are designed with consideration for both the costs and benefits of intermittent augmentation The paper describes the system development and evolution, explorations of usable cognitive mitigation strategies, and four evaluations that show adaptive automaton can effectively, mitigate human decision-making performance at extremes (overload and underload) of workload.

Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment

“…In other recent studies involving ATCOs (Aricò et al, 2013, 2014, 2015b,c; Borghini et al, 2014; Di Flumeri et al, 2015; Toppi et al, 2016), it was demonstrated how it was possible to compute an EEG-based Workload Index able to significantly discriminate the workload demands of the ATM task, and to monitor them continuously by using frontal-parietal brain features. Other studies about the mental workload estimation by using neurophysiological indexes, have been proposed also in other operational contexts (Car drivers - Kohlmorgen et al, 2007; Borghini et al, 2012a; military domain - Dorneich et al, 2005). …”

Systematic literature review on intent‐driven systems

“…Looking at the papers that do fully articulate the proposed methods for expressing and constructing human actors' (level 2), all papers present a theory and an experiment. Dorneich et al [22] focus on the mental state of humans utilising neural networks, while Memon et al [28] are using natural language processing. Bollati et al [16] focus on handling changes supporting model transformations and Topcu [33] uses deliberative coherence theory to handle changes in goals.…”

Provocations: Technology Impedances to Augmented Cognition

“…The U.S. Army conducted another related experiment, which focused on developing a mobile cognitive state classification test bed (Dorneich et al, 2007). This EEG-based system was not drythat is, participants had to wear standard laboratory leads with conducting gel, which were then connected to a laptop computer worn in a backpack.…”