Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction

Yan, Nong; Wang, Chang; Tao, Yi; Li, Jinming; Ke-xu, Zhang; Chen, Ting; Yuan, Ziwen; Yan, Xiao‐Hai; Wang, Gang

doi:10.1109/access.2019.2961246

Cited by 22 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This technology has been shown to improve the lives of numerous patients suffering from various neurological disorders, including amyotrophic lateral sclerosis and spinal cord injuries (Abiri et al, 2019 ; Edelman et al, 2019 ). Over the past few decades, the development of signal acquisition and decoding technology has led to the development of various rehabilitation applications, including neuro-prosthesis (Li et al, 2018a ), wheelchairs (Pinheiro et al, 2018 ), quadcopters (Yan et al, 2020 ), and robotic arms (Cao et al, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm

Liu

et al. 2022

Front. Neurosci.

View full text Add to dashboard Cite

Multiple types of brain-control systems have been applied in the field of rehabilitation. As an alternative scheme for balancing user fatigue and the classification accuracy of brain–computer interface (BCI) systems, facial-expression-based brain control technologies have been proposed in the form of novel BCI systems. Unfortunately, existing machine learning algorithms fail to identify the most relevant features of electroencephalogram signals, which further limits the performance of the classifiers. To address this problem, an improved classification method is proposed for facial-expression-based BCI (FE-BCI) systems, using a convolutional neural network (CNN) combined with a genetic algorithm (GA). The CNN was applied to extract features and classify them. The GA was used for hyperparameter selection to extract the most relevant parameters for classification. To validate the superiority of the proposed algorithm used in this study, various experimental performance results were systematically evaluated, and a trained CNN-GA model was constructed to control an intelligent car in real time. The average accuracy across all subjects was 89.21 ± 3.79%, and the highest accuracy was 97.71 ± 2.07%. The superior performance of the proposed algorithm was demonstrated through offline and online experiments. The experimental results demonstrate that our improved FE-BCI system outperforms the traditional methods.

show abstract

Section: Introductionmentioning

confidence: 99%

A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm

Liu

et al. 2022

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…RAIN computer interface (BCI) is a technology that establishes a communication system between the human brain and external devices [1]. It has been applied in many fields, such as stroke rehabilitation [2,3], prosthetic control [4], quadcopter control [5], speech synthesis [6], and emotion recognition [7]. The techniques aiming to reconstruct hand motor function have been extensively studied [8][9][10] and are expected to enable rehabilitation training for stroke patients.…”

Section: Introductionmentioning

confidence: 99%

Decoding Multi-Class EEG Signals of Hand Movement Using Multivariate Empirical Mode Decomposition and Convolutional Neural Network

Tao

Wang

et al. 2022

IEEE Trans. Neural Syst. Rehabil. Eng.

Self Cite

View full text Add to dashboard Cite

Brain-computer interface (BCI) is a technology that connects the human brain and external devices. Many studies have shown the possibility of using it to restore motor control in stroke patients. One specific challenge of such BCI is that the classification accuracy is not high enough for multi-class movements. In this study, by using Multivariate Empirical Mode Decomposition (MEMD) and Convolutional Neural Network (CNN), a novel algorithm (MECN) was proposed to decode EEG signals for four kinds of hand movements. Firstly, the MEMD was used to decompose the movement-related electroencephalogram (EEG) signals to obtain the multivariate intrinsic empirical functions (MIMFs). Then, the optimal MIMFs fusion was performed based on sequential forward selection algorithm. Finally, the selected MIMFs were input to the CNN model for discriminating four kinds of hand movements. The average classification accuracy of thirteen subjects over the six-fold cross-validation reached 81.14% for 2s-data before the movement onset and 81.08% for 2s-data after

show abstract

“…This technology is widely considered to be a potentially effective approach to reconstruct the basic ability for paralyzed people in activities of daily living (ADL). The development of signal acquisition and decoding technologies have led to the rise of many brain-actuated applications including quadcopter , Yan et al 2020, wheelchair (Tanaka et al 2005, Pinheiro et al 2018, speller output (Chen et al 2015b and computer operation (He et al 2020). Meanwhile, the brain-actuated robotic arm system, as one of the most promising and direct technologies in restoring the motor functions of the upper limbs, plays an important role in the BCI applications.…”

Section: Introductionmentioning

confidence: 99%

A brain-actuated robotic arm system using non-invasive hybrid brain–computer interface and shared control strategy

Cao

Yang

et al. 2021

J. Neural Eng.

View full text Add to dashboard Cite

Objective. The electroencephalography (EEG)-based brain-computer interfaces (BCIs) have been used in the control of robotic arms. The performance of non-invasive BCIs may not be satisfactory due to the poor quality of EEG signals, so the shared control strategies were tried as an alternative solution. However, most of the existing shared control methods set the arbitration rules manually, which highly depended on the specific tasks and developer's experience. In this study, we proposed a novel shared control model that automatically optimized the control commands in a dynamical way based on the context in real-time control. Besides, we employed the hybrid BCI to better allocate commands with multiple functions. The system allowed non-invasive BCI users to manipulate a robotic arm moving in a three-dimensional (3D) space and complete a pick-place task of multiple objects. Approach. Taking the scene information obtained by computer vision as a knowledge base, a machine agent was designed to infer the user's intention and generate automatic commands. Based on the inference confidence and user's characteristic, the proposed shared control model fused the machine autonomy and human intention dynamically for robotic arm motion optimization during the online control. In addition, we introduced a hybrid BCI scheme that applied steady-state visual evoked potentials and motor imagery to the divided primary and secondary BCI interfaces to better allocate the BCI resources (e.g. decoding computing power, screen occupation) and realize the multi-dimensional control of the robotic arm. Main results. Eleven subjects participated in the online experiments of picking and placing five objects that scattered at different positions in a 3D workspace. The results showed that most of the subjects could control the robotic arm to complete accurate and robust picking task with an average success rate of approximately 85% under the shared control strategy, while the average success rate of placing task controlled by pure BCI was 50% approximately. Significance. In this paper, we proposed a novel shared controller for motion automatic optimization, together with a hybrid BCI control scheme that allocated paradigms according to the importance of commands to realize multi-dimensional and effective control of a robotic arm. Our study indicated that the shared control strategy with hybrid BCI could greatly improve the performance of the brain-actuated robotic arm system.

show abstract

Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction

Cited by 22 publications

References 38 publications

A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm

A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm

Decoding Multi-Class EEG Signals of Hand Movement Using Multivariate Empirical Mode Decomposition and Convolutional Neural Network

A brain-actuated robotic arm system using non-invasive hybrid brain–computer interface and shared control strategy

Contact Info

Product

Resources

About