No abstract

A novel hybrid brain-computer interface (BCI) based on the electroencephalogram (EEG) signal which consists of a motor imagery- (MI-) based online interactive brain-controlled switch, “teeth clenching” state detector, and a steady-state visual evoked potential- (SSVEP-) based BCI was proposed to provide multidimensional BCI control. MI-based BCI was used as single-pole double throw brain switch (SPDTBS). By combining the SPDTBS with 4-class SSEVP-based BCI, movement of robotic arm was controlled in three-dimensional (3D) space. In addition, muscle artifact (EMG) of “teeth clenching” condition recorded from EEG signal was detected and employed as interrupter, which can initialize the statement of SPDTBS. Real-time writing task was implemented to verify the reliability of the proposed noninvasive hybrid EEG-EMG-BCI. Eight subjects participated in this study and succeeded to manipulate a robotic arm in 3D space to write some English letters. The mean decoding accuracy of writing task was 0.93 ± 0.03. Four subjects achieved the optimal criteria of writing the word “HI” which is the minimum movement of robotic arm directions (15 steps). Other subjects had needed to take from 2 to 4 additional steps to finish the whole process. These results suggested that our proposed hybrid noninvasive EEG-EMG-BCI was robust and efficient for real-time multidimensional robotic arm control.

No abstract

An extended state observer based fractional order sliding-mode control (ESO-FOSMC) is proposed in this study, with consideration of the strong nonlinear characteristics of a new electro-hydraulic servo system with iso-actuation balancing and positioning. By adopting the fractional order calculus theory, a fractional order proportional-integral-derivative (PID)based sliding mode surface was designed, which has the ability to obtain an equivalent positioning control with fractional order kinetic characteristics. By introducing the integral term into the sliding mode surface, it was found to be beneficial in reducing the steady-state errors, as well as improving the precision of the control system. Also, by using the fractional order calculus to replace the integral calculus, the form of the convergence is improved; the system transfer of energy is slowed down; and the chattering of the system is greatly weakened. The extended state observer was designed to observe the real-time disturbances, and also to generate the compensation control commands which are added to the FOSMC to achieve the dynamic compensation. By means of numerical simulations, the dynamic and static characteristics of the sliding mode control system were compared with those of the FOSMC and ESO-FOSMC. The experimental results show that the ESO-FOSMC system could effectively restrain the external disturbances and achieve higher control precision, as well as better control quantity without chattering. The semi-physical simulations based experimental tests also demonstrated that the proposed ESO-FOSMC outperformed the FOSMC in terms of system robustness and control precision, which could have a stable control of the gun system quickly and accurately.

Motion control of gun barrels is an ongoing topic for the development of gun control equipments possessing excellent performances. In this paper, a typical fractional order PID control strategy is employed for the gun control system. To obtain optimal parameters of the controller, a multiobjective optimization scheme is developed from the loop-shaping perspective. To solve the specified nonlinear optimization problem, a novel Pareto optimal solution based multiobjective differential evolution algorithm is proposed. To enhance the convergent rate of the optimization process, an opposition based learning method is embedded in the chaotic population initialization process. To enhance the robustness of the algorithm for different problems, an adapting scheme of the mutation operation is further employed. With assistance of the evolutionary algorithm, the optimal solution for the specified problem is selected. The numerical simulation results show that the control system can rapidly follow the demand signal with high accuracy and high robustness, demonstrating the efficiency of the proposed controller parameter tuning method.

Recently, the study of emotion recognition has received increasing attentions by the rapid development of noninvasive sensor technologies, machine learning algorithms and compute capability of computers. Compared with single modal emotion recognition, the multimodal paradigm introduces complementary information for emotion recognition. Hence, in this work, we presented a decision level fusion framework for detecting emotions continuously by fusing the Electroencephalography (EEG) and facial expressions. Three types of movie clips (positive, negative, and neutral) were utilized to elicit specific emotions of subjects, the EEG and facial expression signals were recorded simultaneously. The power spectrum density (PSD) features of EEG were extracted by time-frequency analysis, and then EEG features were selected for regression. For the facial expression, the facial geometric features were calculated by facial landmark localization. Long short-term memory networks (LSTM) were utilized to accomplish the decision level fusion and captured temporal dynamics of emotions. The results have shown that the proposed method achieved outstanding performance for continuous emotion recognition, and it yields 0.625±0.029 of concordance correlation coefficient (CCC). From the results, the fusion of two modalities outperformed EEG and facial expression separately. Furthermore, different numbers of time-steps of LSTM was applied to analyze the temporal dynamic capturing.INDEX TERMS Continuous emotion recognition, EEG, facial expressions, signal processing, decision level fusion, temporal dynamics.

Motor imagery is one of the classical paradigms which have been used in brain-computer interface and motor function recovery. Finger movement-based motor execution is a complex biomechanical architecture and a crucial task for establishing most complicated and natural activities in daily life. Some patients may suffer from alternating hemiplegia after brain stroke and lose their ability of motor execution. Fortunately, the ability of motor imagery might be preserved independently and worked as a backdoor for motor function recovery. The efficacy of motor imagery for achieving significant recovery for the motor cortex after brain stroke is still an open question. In this study, we designed a new paradigm to investigate the neural mechanism of thirty finger movements in two scenarios: motor execution and motor imagery. Eleven healthy participants performed or imagined thirty hand gestures twice based on left and right finger movements. The electroencephalogram (EEG) signal for each subject during sixty trials left and right finger motor execution and imagery were recorded during our proposed experimental paradigm. The Granger causality (G-causality) analysis method was employed to analyze the brain connectivity and its strength between contralateral premotor, motor, and sensorimotor areas. Highest numbers for G-causality trials of 37 ± 7.3, 35.5 ± 8.8, 36.3 ± 10.3, and 39.2 ± 9.0 and lowest Granger causality coefficients of 9.1 ± 3.2, 10.9 ± 3.7, 13.2 ± 0.6, and 13.4 ± 0.6 were achieved from the premotor to motor area during execution/imagination tasks of right and left finger movements, respectively. These results provided a new insight into motor execution and motor imagery based on hand gestures, which might be useful to build a new biomarker of finger motor recovery for partially or even completely plegic patients. Furthermore, a significant difference of the G-causality trial number was observed during left finger execution/imagery and right finger imagery, but it was not observed during the right finger execution phase. Significant difference of the G-causality coefficient was observed during left finger execution and imagery, but it was not observed during right finger execution and imagery phases. These results suggested that different MI-based brain motor function recovery strategies should be taken for right-hand and left-hand patients after brain stroke.

Keywords: hybrid brain computer interface (hBCI), common spatial pattern (CSP), hierarchical support vector machine (hSVM)