Use of Force Feedback Device in a Hybrid Brain-Computer Interface Based on SSVEP, EOG and Eye Tracking for Sorting Items

Abstract

Research focused on signals derived from the human organism is becoming increasingly popular. In this field, a special role is played by brain-computer interfaces based on brainwaves. They are becoming increasingly popular due to the downsizing of EEG signal recording devices and ever-lower set prices. Unfortunately, such systems are substantially limited in terms of the number of generated commands. This especially applies to sets that are not medical devices. This article proposes a hybrid brain-computer system based on the Steady-State Visual Evoked Potential (SSVEP), EOG, eye tracking, and force feedback system. Such an expanded system eliminates many of the particular system shortcomings and provides much better results. The first part of the paper presents information on the methods applied in the hybrid brain-computer system. The presented system was tested in terms of the ability of the operator to place the robot’s tip to a designated position. A virtual model of an industrial robot was proposed, which was used in the testing. The tests were repeated on a real-life industrial robot. Positioning accuracy of system was verified with the feedback system both enabled and disabled. The results of tests conducted both on the model and on the real object clearly demonstrate that force feedback improves the positioning accuracy of the robot’s tip when controlled by the operator. In addition, the results for the model and the real-life industrial model are very similar. In the next stage, research was carried out on the possibility of sorting items using the BCI system. The research was carried out on a model and a real robot. The results show that it is possible to sort using bio signals from the human body.

Revisión del uso de interfaces cerebro-computador con el paradigma de Potenciales Evocados Visuales en Estado Estable en el control de robots manipuladores

“…En la Figura 11 se presenta las metodologías utilizadas por los diferentes autores; una de ellas es el análisis de correlación canónica (CCA) (Boboc et al, 2019;Du et al, 2021;Gao et al, 2017;Lin et al, 2019;Yang et al, 2018;Zhang et al, 2017;Zhang et al, 2018) o, variaciones de este método, como el análisis de correlación canónica de banco de filtros (FBCCA) (Cao et al, 2021;Chen et al, 2018;Chen et al, 2020), el método eCCA-Y, desarrollado e implementado por Yin et al (2021) o, el método de análisis canónico de correlación basado en plantillas comunes optimizadas (OCT-CCA) (Peng et al, 2022). Entre otras metodologías, están las máquinas de vector soporte (SVM) (Han et al, 2019;Kubacki, 2021;Li et al, 2019;Rakshit et al, 2017), una de las más utilizadas, clasificadores lineales o clasificadores como la extensión al índice de sincronización multivariable empleado por Ha et al (2021)…”

“…Una interfaz cerebro-computador (BCI) es un sistema que permite la comunicación directa entre el cerebro y una máquina, por lo que también se llama interfaz cerebro-máquina (BMI, por sus siglas en inglés). Esta comunicación permite determinar la intención de un usuario para interactuar con un elemento de su entorno, por lo que puede ser utilizada para diversas aplicaciones como, por ejemplo, el control de drones (Dumitrescu et al, 2021), robots móviles (Liu et al, 2020), exoesqueletos (Gordleeva et al, 2017) o brazos robóticos (Achic et al, 2016;Chen et al, 2020;Du et al, 2021;Ha et al, 2021;Kubacki, 2021;Postelnicu et al, 2019;Rakshit et al, 2016;Yang et al, 2018;Zhang et al, 2017), usados principalmente en tareas de asistencia, permitiendo que personas con parálisis puedan manipular objetos y realizar diferentes tareas de forma independiente.…”

Eye-Tracking Feature Extraction for Biometric Machine Learning

“…In a recent study by Kubacki (2021), the author proposed a novel hybrid BCI system for item sorting. The research is carried out using an industrial robot in the virtual model.…”

Toward a Brain-Computer Interface- and Internet of Things-Based Smart Ward Collaborative System Using Hybrid Signals

“…Currently, motor-impaired individuals and speech-impaired individuals can control a virtual computer cursor [ 8 ] and computer keyboard [ 9 , 10 ] to browse the web by means of noninvasive BCIs. Due to the inconvenience for patients with quadriplegia, many scholars have proposed BCIs based on steady-state visual evoked potential (SSEVP) and motor imagery (MI) to support wheelchair mobility to assist patients in walking [ 11 ], to control robotic arms instead of manual grasping [ 12 – 15 ], to control a robotic arm in place of an artificial arm [ 16 ], and to enable interaction by means of various other assistive devices [ 17 , 18 ]. Although single-mode BCIs have achieved some promising results in the abovementioned studies, hybrid BCIs based on multimodal signals and shared control systems can offer improved accuracy and interaction efficiency [ 19 ].…”