A Hybrid Brain–Computer Interface for Real-Life Meal-Assist Robot Control

Abstract

Assistant devices such as meal-assist robots aid individuals with disabilities and support the elderly in performing daily activities. However, existing meal-assist robots are inconvenient to operate due to non-intuitive user interfaces, requiring additional time and effort. Thus, we developed a hybrid brain–computer interface-based meal-assist robot system following three features that can be measured using scalp electrodes for electroencephalography. The following three procedures comprise a single meal cycle. (1) Triple eye-blinks (EBs) from the prefrontal channel were treated as activation for initiating the cycle. (2) Steady-state visual evoked potentials (SSVEPs) from occipital channels were used to select the food per the user’s intention. (3) Electromyograms (EMGs) were recorded from temporal channels as the users chewed the food to mark the end of a cycle and indicate readiness for starting the following meal. The accuracy, information transfer rate, and false positive rate during experiments on five subjects were as follows: accuracy (EBs/SSVEPs/EMGs) (%): (94.67/83.33/97.33); FPR (EBs/EMGs) (times/min): (0.11/0.08); ITR (SSVEPs) (bit/min): 20.41. These results revealed the feasibility of this assistive system. The proposed system allows users to eat on their own more naturally. Furthermore, it can increase the self-esteem of disabled and elderly peeople and enhance their quality of life.

Enhancing Detection of Control State for High-Speed Asynchronous SSVEP-BCIs Using Frequency-Specific Framework

“…These results suggest that high-speed and high-performance asynchronous SSVEP-BCI systems can be built using the FS framework to adapt to different working contexts. In contrast to studies using physiological signals to implement switches [13][14][15][16][17][18][19][20], the method in this study requires no additional action to transfer between IC and NC states. It is a more convenient and friendly way to implement asynchronous SSVEP-BCIs.…”

“…Several hBCI studies that employed electrooculogram (EOG) [13][14][15][16], electromyogram (EMG) [16,17], P300 [18,19], and respiration signals [20] to establish switches for SSVEP-BCIs have shown their potential to construct asynchronous SSVEP-BCIs for practical applications. In these studies, the users are usually required to perform intentional actions like eye blinks, occlusal movements, P300 tasks, and breath-holding to produce expected patterns to trigger the switch.…”

Eye-Tracking Feature Extraction for Biometric Machine Learning

“…In a study by Guo et al (2019), the authors utilized blink and pupil diameter for five-class emotion recognition with the combination of EEG and EIG. In a study by Ha et al (2021), the authors created a meal assistant device to improve the self-esteem and enhance the life quality of disabled and elderly people. The proposed brain-computer interface (BCI) system is developed based on the features of triple eye blinks, EEG, and EMG.…”

Visual-based feeding intention recognition for feeding robots

“…In the scenario of feeding, ensuring the safety of human-robot interaction and accurately recognizing the user's intention are critical tasks. In South Korea, Jihyeon Ha, and Sangin Park have developed a braincomputer interface system based on electroencephalography (EEG), allowing users to operate devices according to their intentions, resulting in a more natural process for dining [9]. However, this approach may entail inconvenience due to the necessity of wearing associated equipment.…”