Adaptive asynchronous control system of robotic arm based on augmented reality-assisted brain–computer interface

Abstract

Objective. Brain-controlled robotic arms have shown broad application prospects with the development of robotics, science and information decoding. However, disadvantages, such as poor flexibility restrict its wide application. Approach. In order to alleviate these drawbacks, this study proposed a robotic arm asynchronous control system based on steady-state visual evoked potential (SSVEP) in an augmented reality (AR) environment. In the AR environment, the participants were able to concurrently see the robot arm and visual stimulation interface through the AR device. Therefore, there was no need to switch attention frequently between the visual stimulation interface and the robotic arm. This study proposed a multi-template algorithm based on canonical correlation analysis and task-related component analysis to identify 12 targets. An optimization strategy based on dynamic window was adopted to adjust the duration of visual stimulation adaptively. Main results. Experimental results of this study found that the high-frequency SSVEP-based brain–computer interface (BCI) realized the switch of the system state, which controlled the robotic arm asynchronously. The average accuracy of the offline experiment was 94.97 % , whereas the average information translate rate was 67.37 ± 14.27 bits·min−1. The online results from ten healthy subjects showed that the average selection time of a single online command was 2.04 s, which effectively reduced the visual fatigue of the subjects. Each subject could quickly complete the puzzle task. Significance. The experimental results demonstrated the feasibility and potential of this human-computer interaction strategy and provided new ideas for BCI-controlled robots.

Enhancing Detection of Multi-Frequency-Modulated SSVEP Using Phase Difference Constrained Canonical Correlation Analysis

“…Last but not least, the existing study is only dedicated to the synchronous BCIs, which cannot achieve the autonomous control of the external equipment. The following work could extend it to the asynchronous mode, like in [45], [46], which can greatly increase the user's autonomy in the real-life applications [47].…”

Augmented Reality Driven Steady-State Visual Evoked Potentials for Wheelchair Navigation

“…The combination of the BCI with the eyetracker yielded a comparable accuracy of 89.3%, however a major usability concern and cost issue arises due to the discomfort and cost of using and buying two headsets. Finally, the authors in [38] incorporated SSVEP targets in AR glasses to control a robotic arm. The average accuracy of the offline experiment was 94.97% but a direct comparison with our system would be misleading, since during the navigation of a wheelchair the user is moving with the wheelchair and this movement certainly affects the quality of the EEG signals, and consequently the general performance of the system.…”

An extended binary subband canonical correlation analysis detection algorithm oriented to the radial contraction-expansion motion steady-state visual evoked paradigm

“…In addition, the psychological status and environmental changes of subjects are also the influencing factors of the optimal window length. The dynamic window method can effectively reduce the above adverse effects and dynamically adjust the data length required by the algorithm while maintaining the high precision of the decision to achieve a higher ITR, thereby improving the performance of the BCI [23].…”