Automated Task Load Detection with Electroencephalography: Towards Passive Brain–Computer Interfacing in Robotic Surgery

Abstract

Automatic detection of the current task load of a surgeon in the theatre in real time could provide helpful information, to be used in supportive systems. For example, such information may enable the system to automatically support the surgeon when critical or stressful periods are detected, or to communicate to others when a surgeon is engaged in a complex maneuver and should not be disturbed. Passive brain-computer interfaces (BCI) infer changes in cognitive and affective state by monitoring and interpreting ongoing brain activity recorded via an electroencephalogram. The resulting information can then be used to automatically adapt a technological system to the human user. So far, passive BCI have mostly been investigated in laboratory settings, even though they are intended to be applied in real-world settings. In this study, a passive BCI was used to assess changes in task load of skilled surgeons performing both simple and complex surgical training tasks. Results indicate that the introduced methodology can reliably and continuously detect changes in task load in this realistic environment.

In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

“…Passive BCIs (pBCI) are of particular interest for neuroergonomic applications (Cutrell and Tan, 2008 ; Frey et al, 2017 ; Gramann et al, 2017 ). They allow the use of interpretation of unlabeled brain activity during a task to derive various mental states (Blankertz et al, 2010 ; Roy et al, 2013 ; Van Erp et al, 2015 ; Zander et al, 2017 ). These mental state-inference systems offer a unique insight into the development of the human-system interactions to overcome cognitive limitations (Zander and Kothe, 2011 ; Brouwer et al, 2013 ).…”

Brain–Computer Interface Contributions to Neuroergonomics

“…More particularly, they are used for different purposes, the primary one being the evaluation of a product, a work setting or a work task, in order to determine their usability, performance, and generally their impact on the user. Hence, passive BCIs can be used for the evaluation of the comfort of stereoscopic displays (Frey et al, 2016), for the evaluation of the difficulty of a game (Allison & Polich, 2008), a multitasking environment , a flying task (Dehais et al, 2016) or a surgical training procedure (Zander et al, 2016), and also for the evaluation of prolonged and monotonous tasks such as driving (Yeo et al, 2009). Another way of using these systems is to perform an evaluation of the user him/herself, for instance to determine his/her fitness to perform a coming task, or to determine his/her learning type.…”

Independent Decision Path Fusion for Bimodal Asynchronous Brain–Computer Interface to Discriminate Multiclass Mental States

“…For example, through imaginary of left or right hand movement, the amputees can control the movement of corresponding artificial prosthesis as long as their mental states can be accurately detected. Therefore, this powerful tool is nowadays closely associated with numerous healthcare related applications [1]- [6]. In particular, BCI technology is practiced in neurological rehabilitation, where it is applied to help restore motor or cognition functions of patients,…”

“…In motor neuron disease (MND) patients, BCI-based wheelchairs [4] and robotic arms [5] are expected to provide convenience in daily life. Besides, the BCI for automated task load monitoring in robotic surgery was developed [6].…”