Quadcopter Flight Control Using a Non-invasive Multi-Modal Brain Computer Interface

Duan, Xu; Xie, Songyun; Xie, Xinzhou; Meng, Ya; Zhao, Xu

doi:10.3389/fnbot.2019.00023

Cited by 41 publications

(25 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In comparison with the previous relevant research [ 29 , 30 , 44 , 45 , 46 , 47 ] (listed in Table 3 ), instead of using the eye-tracking to detect the eye movement, this study applied the eye blinking to be a trigger in this control system. In the eye-tracking system, the extra device is required to monitor the eye movement.…”

Section: Discussionmentioning

confidence: 99%

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

Yang

Nguyen

2020

Sensors

View full text Add to dashboard Cite

The goal of this study was to develop and validate a hybrid brain-computer interface (BCI) system for home automation control. Over the past decade, BCIs represent a promising possibility in the field of medical (e.g., neuronal rehabilitation), educational, mind reading, and remote communication. However, BCI is still difficult to use in daily life because of the challenges of the unfriendly head device, lower classification accuracy, high cost, and complex operation. In this study, we propose a hybrid BCI system for home automation control with two brain signals acquiring electrodes and simple tasks, which only requires the subject to focus on the stimulus and eye blink. The stimulus is utilized to select commands by generating steady-state visually evoked potential (SSVEP). The single eye blinks (i.e., confirm the selection) and double eye blinks (i.e., deny and re-selection) are employed to calibrate the SSVEP command. Besides that, the short-time Fourier transform and convolution neural network algorithms are utilized for feature extraction and classification, respectively. The results show that the proposed system could provide 38 control commands with a 2 s time window and a good accuracy (i.e., 96.92%) using one bipolar electroencephalogram (EEG) channel. This work presents a novel BCI approach for the home automation application based on SSVEP and eye blink signals, which could be useful for the disabled. In addition, the provided strategy of this study—a friendly channel configuration (i.e., one bipolar EEG channel), high accuracy, multiple commands, and short response time—might also offer a reference for the other BCI controlled applications.

show abstract

Section: Discussionmentioning

confidence: 99%

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

Yang

Nguyen

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The quadcopter used in the system is Parrot Bebop 2 due to the sustainable development and strong stabilization [6], [7], [10], [11], [17], [18]. In the proposed system, eight EEG-based control commands transmitted via the wireless network to control the quadcopter flight in 3D space.…”

Section: ) Quadcopter Systemmentioning

confidence: 99%

“…Kim et al accomplished the eight angles of quadcopter flight control by the combination of EEG signals and eye-tracking [17]. Khan et al proposed a control system based on eight commands by using EEG signals and nearinfrared spectral signals [18]. The hybrid BCI based on the multi-signals was similar to the SSVEP-based BCI which lacks the correlation between signals and users' intentions [19].…”

Section: Introductionmentioning

confidence: 99%

Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction

et al. 2020

View full text Add to dashboard Cite

Quadcopter is an important way for the human to explore the physical world. The brain-computer interface (BCI) technology is used to control the quadcopter flight in order to help disabled persons communicate with the external world freely. In this study, a quadcopter control system using a hybrid BCI based on off-line optimization and enhanced human-machine interaction was designed to control the quadcopter flight in 3D physical space. The proposed system implemented the control of quadcopter moving up/down, forward/backward, left/right by six different SSVEP, and turning left/right by left-hand and right-hand motor imagery. Meanwhile, the optimization of the control system and the human-machine interaction enhancement improved practicability in real-time use. Five subjects participated in an on-line experiment to control the quadcopter flight in real-time. The average classification accuracy of EEG-based commands in the on-line experiment was 87.09±2.82% and information transfer rate (ITR) was 0.857±0.085 bits/min. The results demonstrated the feasibility of multidirectional control of quadcopter flight in 3D space by using hybrid BCI technology and revealed the practicality and operability of the hybrid BCI control system based on off-line optimization and human-machine interaction enhancement. INDEX TERMS Quadcopter control system, motor imagery, steady-state visual evoked potential, off-line optimization, human-machine interaction.

show abstract

“…Traditional methods, such as machine learning and signal processing, are widely used in the study of signal processing for MI-EEG decoding [ 8 , 9 , 10 , 11 , 12 ]. In general, this process consists of the following steps: First, noise and irrelevant frequency bands are removed by preprocessing.…”

Section: Introductionmentioning

confidence: 99%

Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network

Zhang

Han

et al. 2020

Sensors

View full text Add to dashboard Cite

As an important paradigm of spontaneous brain-computer interfaces (BCIs), motor imagery (MI) has been widely used in the fields of neurological rehabilitation and robot control. Recently, researchers have proposed various methods for feature extraction and classification based on MI signals. The decoding model based on deep neural networks (DNNs) has attracted significant attention in the field of MI signal processing. Due to the strict requirements for subjects and experimental environments, it is difficult to collect large-scale and high-quality electroencephalogram (EEG) data. However, the performance of a deep learning model depends directly on the size of the datasets. Therefore, the decoding of MI-EEG signals based on a DNN has proven highly challenging in practice. Based on this, we investigated the performance of different data augmentation (DA) methods for the classification of MI data using a DNN. First, we transformed the time series signals into spectrogram images using a short-time Fourier transform (STFT). Then, we evaluated and compared the performance of different DA methods for this spectrogram data. Next, we developed a convolutional neural network (CNN) to classify the MI signals and compared the classification performance of after DA. The Fréchet inception distance (FID) was used to evaluate the quality of the generated data (GD) and the classification accuracy, and mean kappa values were used to explore the best CNN-DA method. In addition, analysis of variance (ANOVA) and paired t-tests were used to assess the significance of the results. The results showed that the deep convolutional generative adversarial network (DCGAN) provided better augmentation performance than traditional DA methods: geometric transformation (GT), autoencoder (AE), and variational autoencoder (VAE) (p < 0.01). Public datasets of the BCI competition IV (datasets 1 and 2b) were used to verify the classification performance. Improvements in the classification accuracies of 17% and 21% (p < 0.01) were observed after DA for the two datasets. In addition, the hybrid network CNN-DCGAN outperformed the other classification methods, with average kappa values of 0.564 and 0.677 for the two datasets.

show abstract

Quadcopter Flight Control Using a Non-invasive Multi-Modal Brain Computer Interface

Cited by 41 publications

References 35 publications

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction

Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network

Contact Info

Product

Resources

About