Design of a Multimodal EEG-based Hybrid BCI System with Visual Servo Module

Duan, Feng; Lin, Dongxue; Li, Wenyu; Zhang, Zhao

doi:10.1109/tamd.2015.2434951

Cited by 70 publications

(33 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system will be developed to perceive the environment autonomously and display the optimal behaviors, which would most likely be used in the current situation, on the SSVEP interface. In addition, we will also report the performance of accomplishing the operation task using the behavior-based hierarchical architecture based on other brainwave-based models, e.g., ERPs, MI potentials, and hybrid models [34].…”

Section: Discussionmentioning

confidence: 99%

Behavior-Based SSVEP Hierarchical Architecture for Telepresence Control of Humanoid Robot to Achieve Full-Body Movement

Zhao

Mao

et al. 2017

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-The challenge to telepresence control a humanoid robot through a steady-state visual evoked potential (SSVEP) based model is to rapidly and accurately control full-body movement of the robot because a subject has to synchronously recognize the complex natural environments based on live video feedback and activate the proper mental states by targeting the visual stimuli. To mitigate this problem, this paper presents a behavior-based hierarchical architecture, which coordinates a large number of robot behaviors using only the most effective five stimuli. We defined and implemented fourteen robot behaviors for motion control and object manipulation, which were encoded through the visual stimuli of SSVEPs, and classified them into four behavioral sets. We proposed switch mechanisms in the hierarchical architecture to coordinate these behaviors and control the full-body movement of a NAO humanoid robot. To improve operation performance, we investigated the individual sensitivities of visual stimuli and allocated the stimuli targets according to frequency-responsive properties of individual subjects. We compared different types of walking strategies. The experimental results showed that the behavior-based SSVEP hierarchical architecture enabled the humanoid robot to complete an operation task, including navigating to an object and picking the object up with a fast operation time and a low chance of collision in an environment cluttered with obstacles.Index Terms-Brain-robot interaction (BRI), electroencephalogram (EEG), full-body movement, humanoid robot, steady-state visual evoked potential (SSVEP), telepresence control.

show abstract

Section: Discussionmentioning

confidence: 99%

Behavior-Based SSVEP Hierarchical Architecture for Telepresence Control of Humanoid Robot to Achieve Full-Body Movement

Zhao

Mao

et al. 2017

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…The brain-computer interface (BCI) is a system to control the machine without human peripheral neuromuscular system [1,2]. This can effectively enhance the physical ability of the user [3][4][5].…”

Section: Introductionmentioning

confidence: 99%

The Study of Influence of Sound on Visual ERP-Based Brain Computer Interface

2020

Sensors

View full text Add to dashboard Cite

The performance of the event-related potential (ERP)-based brain–computer interface (BCI) declines when applying it into the real environment, which limits the generality of the BCI. The sound is a common noise in daily life, and whether it has influence on this decline is unknown. This study designs a visual-auditory BCI task that requires the subject to focus on the visual interface to output commands and simultaneously count number according to an auditory story. The story is played at three speeds to cause different workloads. Data collected under the same or different workloads are used to train and test classifiers. The results show that when the speed of playing the story increases, the amplitudes of P300 and N200 potentials decrease by 0.86 μV (p = 0.0239) and 0.69 μV (p = 0.0158) in occipital-parietal area, leading to a 5.95% decline (p = 0.0101) of accuracy and 9.53 bits/min decline (p = 0.0416) of information transfer rate. The classifier that is trained by the high workload data achieves higher accuracy than the one trained by the low workload if using the high workload data to test the performance. The result indicates that the sound could affect the visual ERP-BCI by increasing the workload. The large similarity of the training data and testing data is as important as the amplitudes of the ERP on obtaining high performance, which gives us an insight on how make to the ERP-BCI generalized.

show abstract

“…Khan et al proposed a control system based on eight commands by using EEG signals and nearinfrared spectral signals [18]. The hybrid BCI based on the multi-signals was similar to the SSVEP-based BCI which lacks the correlation between signals and users' intentions [19]. Some studies have demonstrated the feasibility of quadcopter control by combining SSVEP-based BCI and motor imagery-based BCI in the physical world.…”

Section: Introductionmentioning

confidence: 99%

Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction

et al. 2020

View full text Add to dashboard Cite

Quadcopter is an important way for the human to explore the physical world. The brain-computer interface (BCI) technology is used to control the quadcopter flight in order to help disabled persons communicate with the external world freely. In this study, a quadcopter control system using a hybrid BCI based on off-line optimization and enhanced human-machine interaction was designed to control the quadcopter flight in 3D physical space. The proposed system implemented the control of quadcopter moving up/down, forward/backward, left/right by six different SSVEP, and turning left/right by left-hand and right-hand motor imagery. Meanwhile, the optimization of the control system and the human-machine interaction enhancement improved practicability in real-time use. Five subjects participated in an on-line experiment to control the quadcopter flight in real-time. The average classification accuracy of EEG-based commands in the on-line experiment was 87.09±2.82% and information transfer rate (ITR) was 0.857±0.085 bits/min. The results demonstrated the feasibility of multidirectional control of quadcopter flight in 3D space by using hybrid BCI technology and revealed the practicality and operability of the hybrid BCI control system based on off-line optimization and human-machine interaction enhancement. INDEX TERMS Quadcopter control system, motor imagery, steady-state visual evoked potential, off-line optimization, human-machine interaction.

show abstract

Design of a Multimodal EEG-based Hybrid BCI System with Visual Servo Module

Cited by 70 publications

References 28 publications

Behavior-Based SSVEP Hierarchical Architecture for Telepresence Control of Humanoid Robot to Achieve Full-Body Movement

Behavior-Based SSVEP Hierarchical Architecture for Telepresence Control of Humanoid Robot to Achieve Full-Body Movement

The Study of Influence of Sound on Visual ERP-Based Brain Computer Interface

Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction

Contact Info

Product

Resources

About