Virtual and Actual Humanoid Robot Control with Four-Class Motor-Imagery-Based Optical Brain-Computer Interface

Abstract

Motor-imagery tasks are a popular input method for controlling brain-computer interfaces (BCIs), partially due to their similarities to naturally produced motor signals. The use of functional near-infrared spectroscopy (fNIRS) in BCIs is still emerging and has shown potential as a supplement or replacement for electroencephalography. However, studies often use only two or three motor-imagery tasks, limiting the number of available commands. In this work, we present the results of the first four-class motor-imagery-based online fNIRS-BCI for robot control. Thirteen participants utilized upper- and lower-limb motor-imagery tasks (left hand, right hand, left foot, and right foot) that were mapped to four high-level commands (turn left, turn right, move forward, and move backward) to control the navigation of a simulated or real robot. A significant improvement in classification accuracy was found between the virtual-robot-based BCI (control of a virtual robot) and the physical-robot BCI (control of the DARwIn-OP humanoid robot). Differences were also found in the oxygenated hemoglobin activation patterns of the four tasks between the first and second BCI. These results corroborate previous findings that motor imagery can be improved with feedback and imply that a four-class motor-imagery-based fNIRS-BCI could be feasible with sufficient subject training.

In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

“…We also compared pilot's WM performance and related PFC activity both in high fidelity simulator and real flight conditions. The objective was to determine wether these two conditions simulated and real operational settings were equivalent or not in terms of task demand (Dahlstrom and Nahlinder, 2009 ; Batula et al, 2017 ). As most aviation psychology experiments and pilots' training are conducted with flight simulators, such assessment is critical for future design and development of such approaches (Philip et al, 2005 ).…”

Application of functional near-infrared spectroscopy in the healthcare industry: A review

“…364 The neuronal activity generation due to motor imagery was found to be enhanced with visual feedback during a robot movement control using motor imagery. 365 The acquisition of data from a realistic environment and its processing are vital for the development of BCI. 366,367 Research on the mental states and neuroergonomics of pilots during actual as well as simulated environment was conducted using fNIRS.…”

Functional Near Infrared Spectroscopy: Enabling routine functional brain imaging

“…In a recent work, Chaudhary and colleagues have succeeded to reach above chance level accuracy to a yes/no paradigm with patients who are in a complete locked-in state [52]. Another interesting study attempted to control a robot through mere motor imagery [53] comparable to what has been done with EEG [54]. This type of research can have a high impact on prosthetics such as controlling a robot-arm to perform daily functions, controlling a wheelchair or the environment without the need of a chip implant.…”