Continuous Hybrid BCI Control for Robotic Arm Using Noninvasive Electroencephalogram, Computer Vision, and Eye Tracking

Abstract

The controlling of robotic arms based on brain–computer interface (BCI) can revolutionize the quality of life and living conditions for individuals with physical disabilities. Invasive electroencephalography (EEG)-based BCI has been able to control multiple degrees of freedom (DOFs) robotic arms in three dimensions. However, it is still hard to control a multi-DOF robotic arm to reach and grasp the desired target accurately in complex three-dimensional (3D) space by a noninvasive system mainly due to the limitation of EEG decoding performance. In this study, we propose a noninvasive EEG-based BCI for a robotic arm control system that enables users to complete multitarget reach and grasp tasks and avoid obstacles by hybrid control. The results obtained from seven subjects demonstrated that motor imagery (MI) training could modulate brain rhythms, and six of them completed the online tasks using the hybrid-control-based robotic arm system. The proposed system shows effective performance due to the combination of MI-based EEG, computer vision, gaze detection, and partially autonomous guidance, which drastically improve the accuracy of online tasks and reduce the brain burden caused by long-term mental activities.

Subject-Dependent Artifact Removal for Enhancing Motor Imagery Classifier Performance under Poor Skills

“…One more issue of consideration is to explore the effectiveness of the developed approach for artifact removal in modern architectures of deep learning [ 15 ]. Moreover, eye-tracking and cognitive psychological attention test data could be of benefit to enhance our artifact removal approach and favor both BCI performance and interpretability [ 66 , 67 ].…”

Manipulación de un brazo robot mediante el seguimiento de los ojos

“…La principal aportación de este artículo se sitúa en la construcción de un prototipo o plataforma experimental de seguimiento ocular que captura la información del dispositivo de seguimiento ocular y la manipulación de un robot a distancia. En (Xu, et. al.…”

Manual 3D Control of an Assistive Robotic Manipulator Using Alpha Rhythms and an Auditory Menu: A Proof-of-Concept

“…The proposed BCI provides users with flexible manual end-point control of an ARM; yet, compared to interfaces with shared control [29,43] or goal selection control [39,41,44], its task throughput is much lower. Depending on the specificity of the actions provided, goal selection strategies will offer vastly different experiences to the user.…”

“…For example, Lillo et al [41] proposed a P300-based BCI capable of performing drinking and object manipulation tasks using commands comprised of several automated actions. Indeed, many state-of-the-art approaches consist of combining BCIs with computer vision-based object recognition to obtain fast autonomous control during target manipulation [29,[42][43][44]. While these automated strategies are often faster and less fatiguing than manual end-point control, they cannot be easily implemented in unknown environments.…”