A Novel MI-EEG Imaging With the Location Information of Electrodes

Abstract

Combination of the Motor Imagery EEG (MI-EEG) imaging and Deep Convolutional Neural Network is a prospective recognition method in brain computer interface. Nowadays, the frequency or timefrequency analysis has been applied to each channel of MI-EEG signal to obtain a spatio-frequency or timefrequency image, and even the images from several channels are infused to generate a combined image. However, the real position information of channels or electrodes is lost in these MI-EEG images, and this is contradictory to the activation area of MI-tasks. In this paper, the MI period and the frequency band covered by µ and β rhythms are divided into ten time windows and three sub-bands, respectively. Then, for each electrode, Fast Fourier Transform (FFT) is employed to transform each time window to spectrum, and its inverse FFT is calculated for each sub-band. The time-domain powers of ten time windows are averaged for the same sub-band. So, three average powers are generated as the time-frequency features of each electrode of MI-EEG. They are further arranged to the electrode coordinate figure by using Clough-Tocher interpolation algorithm, and a complicated image, in which the time-frequency features are correctly located at the real position of each electrode, is obtained to embody the MI-EEG in detail. Furthermore, a VGG network is modified to perform effective recognition for MI-EEG image, and it is called mVGG. Extensive experiments are conducted on three publicly available datasets, and the 10-folds cross validation accuracies of 88.62%, 92.28% and 96.86% are achieved respectively, and they are higher than that of the state-of-the-art imaging methods. Kappa values and ROC curves demonstrate our method has lower class skew and error costs. The experimental results show that the effectiveness of proposed MI-EEG imaging method, and it is well-matched with mVGG. INDEX TERMS Brain computer interface, convolutional neural network, interpolation method, machine learning, MI-EEG imaging method.

A Comprehensive Review on Critical Issues and Possible Solutions of Motor Imagery Based Electroencephalography Brain-Computer Interface

“…Li et al [ 117 ] used the TPCT imaging method to fix the electrode positions and assigned time-frequency feature values to each pixel in the MI-EEG image. This way promotes feature fusion from the time, space, and frequency domains, respectively.…”

A Dynamic Multi-Scale Network for EEG Signal Classification

“…Though it has been improved into vector autoregressive (VAR) modeling, this method was not always effective when encountering an unstable sequence (Haboub et al, 2020 ). As for the frequency-domain analysis, the Fast Fourier transform (FFT) and Welch's method were both widely used in this field (Oikonomou et al, 2017 ; Li et al, 2020 ). Compared to FFT, Welch's method reduced the noise information of the original data but offered lower frequency resolution.…”

“…In recent years, networks like the CNN, RNN, stacked autoencoders (SAE), deep belief networks (DBN), and VGGNet (Visual Geometry Group) were widely used in MI EEG systems (Schirrmeister et al, 2017 ; Tang et al, 2017 ; Li et al, 2020 ). These neural networks can complete all the above-mentioned steps because the network layers can extract feature maps from original data and learn to classify according to training labels.…”

“…As one of the most popular networks in MI EEG systems, the CNN was often combined with other techniques such as the WT and STFT in practical experiments. For instance, (Li et al, 2020 ) segmented the EEG data by time windows then employed FFT to transform each time window to spectrum. By using the modified VGG called mVGG, a complicated image containing time-frequency features was generated, and its accuracy reached 88.62, 92.28, and 96.86% on three datasets-higher than that of the state-of-the-art imaging methods (Li et al, 2020 ).…”

“…For instance, (Li et al, 2020 ) segmented the EEG data by time windows then employed FFT to transform each time window to spectrum. By using the modified VGG called mVGG, a complicated image containing time-frequency features was generated, and its accuracy reached 88.62, 92.28, and 96.86% on three datasets-higher than that of the state-of-the-art imaging methods (Li et al, 2020 ). In addition, Chaudhary et al introduced STFT and continuous wavelet transform (CWT) into CNN and drew the conclusion that the CWT approach yields better results than the other existing methods with accuracy score of 99.35% (Chaudhary et al, 2019 ).…”