The Brain Is Faster than the Hand in Split-Second Intentions to Respond to an Impending Hazard: A Simulation of Neuroadaptive Automation to Speed Recovery to Perturbation in Flight Attitude

Abstract

The goal of this research is to test the potential for neuroadaptive automation to improve response speed to a hazardous event by using a brain-computer interface (BCI) to decode perceptual-motor intention. Seven participants underwent four experimental sessions while measuring brain activity with magnetoencephalograpy. The first three sessions were of a simple constrained task in which the participant was to pull back on the control stick to recover from a perturbation in attitude in one condition and to passively observe the perturbation in the other condition. The fourth session consisted of having to recover from a perturbation in attitude while piloting the plane through the Grand Canyon constantly maneuvering to track over the river below. Independent component analysis was used on the first two sessions to extract artifacts and find an event related component associated with the onset of the perturbation. These two sessions were used to train a decoder to classify trials in which the participant recovered from the perturbation (motor intention) vs. just passively viewing the perturbation. The BCI-decoder was tested on the third session of the same simple task and found to be able to significantly distinguish motor intention trials from passive viewing trials (mean = 69.8%). The same BCI-decoder was then used to test the fourth session on the complex task. The BCI-decoder significantly classified perturbation from no perturbation trials (73.3%) with a significant time savings of 72.3 ms (Original response time of 425.0–352.7 ms for BCI-decoder). The BCI-decoder model of the best subject was shown to generalize for both performance and time savings to the other subjects. The results of our off-line open loop simulation demonstrate that BCI based neuroadaptive automation has the potential to decode motor intention faster than manual control in response to a hazardous perturbation in flight attitude while ignoring ongoing motor and visual induced activity related to piloting the airplane.

Disruption in neural phase synchrony is related to identification of inattentional deafness in real‐world setting

“…These results contribute to a growing number of neuroergonomic‐based studies that maintain that the brain can better be understood by investigating it in real world like settings rather than simplified isolated conditions that only occur in the laboratory (Adamson et al, ; Callan et al, ; Callan, Falcone, Wada, & Parasuraman, ; Callan, Terzibas, Cassel, Sato, & Parasuraman, ; Durantin, Scannella, Gateau, Delorme, & Dehais, ; Durantin et al, ; Gateau, Durantin, Lancelot, Scannella, & Dehais, ; Scholl et al, ). This is specifically important when the critical situations under study cannot be easily simulated in the laboratory.…”

A Neuroergonomics Approach to Mental Workload, Engagement and Human Performance

“…Gateau et al (2016) implemented an online attentional state estimator coupled with a stochastic decision framework to dynamically adapt authority sharing between human and robots in a search and rescue scenario to prevent effort withdrawal on the part of the human. In a more extreme fashion, Callan et al (2016) revealed that it is possible to decode user motor intention so automation can perform on behalf of the user to drastically reduce the response time in emergency situations (e.g., collision with terrain). In the future, it is assumed that aircraft designers will implement adaptive automation technology that takes over from the pilots by either inhibiting their inputs on the flight deck or performing automated evasive actions (e.g., automatic pullup) to prevent from perseveration.…”

In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

“…These mental state-inference systems offer a unique insight into the development of the human-system interactions to overcome cognitive limitations (Zander and Kothe, 2011 ; Brouwer et al, 2013 ). While several pBCIs have been successfully implemented in driving (Dijksterhuis et al, 2013 ) and flight simulator (Gateau et al, 2015 ; Aricò et al, 2016 ; Çakır et al, 2016 ; Callan et al, 2016 ; Verdière et al, 2018 ), few have attempted to test these systems under more realistic settings. However, very few studies have attempted to test these adaptive systems under realistic settings (Callan et al, 2015 ).…”