A Real-Time Brain Control Method Based on Facial Expression for Prosthesis Operation

Lu, Zhufeng; Zhang, Xiaodong; Li, Hanzhe; Li, Rui; Chen, Jiangcheng

doi:10.1109/robio.2018.8664724

“…As for the location of the EEG generator, since the precise positioning cannot be achieved without the extra professional equipment, the dipole was set in accordance with the mechanism. In our serial research on ME-BCI (Zhang et al, 2016(Zhang et al, , 2021bLi et al, 2018b;Lu et al, 2018aLu et al, ,b, 2020, data-driven brain connectivity analysis demonstrated the main involvement of the motor cortex (Lu et al, 2018a;Zhang et al, 2021b), which conformed to the contralateral control facts. Meanwhile, evidence showed the frontal lobe and limbic system also participate in facial-expression processing (Price and Drevets, 2010;Li et al, 2018b;Lu et al, 2018b).…”

Section: The Head Models and The Dipole Positionsupporting

confidence: 63%

“…According to our previous experimental setup in the BCI serial studies ( Lu et al, 2018a , 2020 ; Zhang et al, 2021b ), to realize the real-time decoding, continuous collected EEGs (4 s) were sliced into short-windowed (100 ms) to form the training-set. To reproduce the dataset forming procedure and augment the real EEGs samples, the output of the Generator was sliced to window-length size before entering the Discriminator.…”

Section: Methodsmentioning

confidence: 99%

Realizing the Application of EEG Modeling in BCI Classification: Based on a Conditional GAN Converter

Zhang

¹

,

Lu

²

,

Zhang

³

et al. 2021

Front. Neurosci.

Self Cite

4

0

View full text Add to dashboard Cite

Electroencephalogram (EEG) modeling in brain-computer interface (BCI) provides a theoretical foundation for its development. However, limited by the lack of guidelines in model parameter selection and the inability to obtain personal tissue information in practice, EEG modeling in BCI is mainly focused on the theoretical qualitative level which shows a gap between the theory and its application. Based on such problems, this work combined the surface EEG simulation with a converter based on the generative adversarial network (GAN), to establish the connection from simulated EEG to its application in BCI classification. For the scalp EEGs modeling, a mathematical model was built according to the physics of surface EEG, which consisted of the parallel 3-population neural mass model, the equivalent dipole, and the forward computation. For application, a converter based on the conditional GAN was designed, to transfer the simulated theoretical-only EEG to its practical version, in the lack of individual bio-information. To verify the feasibility, based on the latest microexpression-assisted BCI paradigm proposed by our group, the converted simulated EEGs were used in the training of BCI classifiers. The results indicated that, compared with training with insufficient real data, by adding the simulated EEGs, the overall performance showed a significant improvement (P = 0.04 < 0.05), and the test performance can be improved by 2.17% ± 4.23, in which the largest increase was up to 12.60% ± 1.81. Through this work, the link from theoretical EEG simulation to BCI classification has been initially established, providing an enhanced novel solution for the application of EEG modeling in BCI.

show abstract

“…Considering of the low frequency solution resulted from the short time window, feature engineering was realized by a classic spatial filtering method: Common Spatial Pattern (CSP) [21]. The CSP features were calculated as:…”

Section: Step B: Interface 'On' Detectionmentioning

confidence: 99%

microFE-BCI: An Asynchronous Practical EEG-based Control Paradigm Assisted by Micro-facial-expressions

Lu¹

2021

Preprint

Self Cite

1

4

0

View full text Add to dashboard Cite

<div><p>In this work, an EEG-based control paradigm assisted by micro-facial-expressions (microFE-BCI) was developed, focusing on the mainstream defect as the insufficiency of real-time capability, asynchronous logic, and robustness. The core algorithm in microFE-BCI contained two stages (asynchronous ‘ON’ detection & microFE-BCI based real-time control) with four steps (obvious non-microFE-EEGs exclusion, interface ‘ON’ detection, microFE-EEGs real-time decoding, and validity judgment). It provided the asynchrounous function, decoded 8 instructions from the latest 100 ms EEGs, and greatly reduced the frequent misoperation. In the offline assessment, microFE-BCI achieved 96.46%±1.07 accuracy for interface ‘ON' detection and 92.68%±1.21 for microFE-EEGs real-time decoding, with the theoretical output timespan less than 200 ms. This microFE-BCI was implemented into a software, and applied to two online manipulations for evaluating the stability and agility. In object-moving with a robotic arm, the averaged IoU was 60.03±11.53%. In water-pouring with a prosthetic Hand, the averaged water volume was 202.5±7.0 ml. During online, microFE-BCI performed no significant difference (P = 0.6521 & P = 0.7931) with commercial control methods (i.e., FlexPendant and Joystick), indicating a similar level of controllability and agility. This study demonstrated the capability of microFE-BCI, enabling a novel solution to the noninvasive BCIs in real-world challenges.</p></div>

show abstract

“…Compared to the former FE-BCI [21], by limiting the range of facial muscle movement to the micro level, this microFE-BCI greatly reduces the proportion of EMG artifacts and makes it possible to distinguish between daily facial-expressions and microexpressions in EEG-based control. The contrast of EMG artifacts proportion is particularly obvious in the high frequency of EEG.…”

Section: Offline Assessment  Step A: Obvious Non-microfe-eegs Exclusionmentioning

confidence: 99%

“…Moreover, it needs to satisfy the basic requirements of real-time, precision, user-friendliness, and easiness, similar to the traditional control approaches. An EEG-based control paradigm assisted by facial-expression (FE-BCI) proposed by our research group [21][22][23][24][25] provides the capability for real-time decoding and control (each output generated from the latest 100 ms EEGs) [21], and has the characteristics of no additional user adaption, no stimulators, and no nerve adaptability [23].…”

Section: Introductionmentioning

confidence: 99%

microFE-BCI: An Asynchronous Practical EEG-based Control Paradigm Assisted by Micro-facial-expressions

Lu¹

2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

<div><p>In this work, an EEG-based control paradigm assisted by micro-facial-expressions (microFE-BCI) was developed, focusing on the mainstream defect as the insufficiency of real-time capability, asynchronous logic, and robustness. The core algorithm in microFE-BCI contained two stages (asynchronous ‘ON’ detection & microFE-BCI based real-time control) with four steps (obvious non-microFE-EEGs exclusion, interface ‘ON’ detection, microFE-EEGs real-time decoding, and validity judgment). It provided the asynchrounous function, decoded 8 instructions from the latest 100 ms EEGs, and greatly reduced the frequent misoperation. In the offline assessment, microFE-BCI achieved 96.46%±1.07 accuracy for interface ‘ON' detection and 92.68%±1.21 for microFE-EEGs real-time decoding, with the theoretical output timespan less than 200 ms. This microFE-BCI was implemented into a software, and applied to two online manipulations for evaluating the stability and agility. In object-moving with a robotic arm, the averaged IoU was 60.03±11.53%. In water-pouring with a prosthetic Hand, the averaged water volume was 202.5±7.0 ml. During online, microFE-BCI performed no significant difference (P = 0.6521 & P = 0.7931) with commercial control methods (i.e., FlexPendant and Joystick), indicating a similar level of controllability and agility. This study demonstrated the capability of microFE-BCI, enabling a novel solution to the noninvasive BCIs in real-world challenges.</p></div>

show abstract

A Real-Time Brain Control Method Based on Facial Expression for Prosthesis Operation

Cited by 6 publications

References 17 publications

Realizing the Application of EEG Modeling in BCI Classification: Based on a Conditional GAN Converter

Realizing the Application of EEG Modeling in BCI Classification: Based on a Conditional GAN Converter

microFE-BCI: An Asynchronous Practical EEG-based Control Paradigm Assisted by Micro-facial-expressions

microFE-BCI: An Asynchronous Practical EEG-based Control Paradigm Assisted by Micro-facial-expressions

Contact Info

Product

Resources

About