High-Speed Spelling in Virtual Reality with Sequential Hybrid BCIs

Yao, Zhaolin; Ma, Xinyao; Wang, Yijun; Zhang, Xu; Liu, Ming; Chen, Hongda

doi:10.1587/transinf.2018edl8122

Cited by 16 publications

(17 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the optimal data length for each trial can be different, an adaptive selection of time window length using a dynamic stopping criteria can be a solution for BCI users [37][38]. Besides, the combination of SSVEP and other modalities, e.g., eye-tracking systems [39] can improve the performance compared with using two single-modality methods. However, efficiency of hybrid methods over single-modality methods needs to be investigated.…”

Section: Discussionmentioning

confidence: 99%

Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs

Mehdizavareh¹,

Hemati²,

Soltanian-Zadeh³

2020

PLoS ONE

View full text Add to dashboard Cite

Recently, steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) has attracted much attention due to its high information transfer rate (ITR) and increasing number of targets. However, the performance of SSVEP-based methods in terms of accuracy and time length required for target detection can be improved. We propose a new canonical correlation analysis (CCA)-based method to integrate subject-specific models and subject-independent information and enhance BCI performance. To optimize hyperparameters for CCA-based model of a specific subject, we propose to use training data of other subjects. An ensemble version of the proposed method is also developed and used for a fair comparison with ensemble task-related component analysis (TRCA). A publicly available 35-subject SSVEP benchmark dataset is used to evaluate different methods. The proposed method is compared with TRCA and extended CCA methods as reference methods. The performance of the methods is evaluated using classification accuracy and ITR. Offline analysis results show that the proposed method reaches highest ITR compared with TRCA and extended CCA. Also, the proposed method significantly improves performance of extended CCA in all conditions and TRCA for time windows greater than 0.3 s. In addition, the proposed method outperforms TRCA for low number of training blocks and electrodes. This study illustrates that adding subject-independent information to subject-specific models can improve the performance of SSVEP-based BCIs.

show abstract

Section: Discussionmentioning

confidence: 99%

Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs

Mehdizavareh¹,

Hemati²,

Soltanian-Zadeh³

2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Some studies have proposed a more flexible and dynamic paradigm [ 99 , 100 , 101 ] that involves extracting a small region by first detecting fuzzy areas of the user’s gaze. The system then encodes the targets in the extracted region by frequency to evoke the SSVEP and uses it to detect the target in the region at which the user is staring.…”

Section: Ssvep-based Hybrid Bci Spellersmentioning

confidence: 99%

“…Many SSVEP-based hybrid spellers have been proposed, and have delivered better performance than any single triggering method-based speller. Section 4 has provided several triggering methods that have been combined with the SSVEP to form a hybrid speller, including the visual ERP P300 [ 84 , 85 , 86 , 87 , 88 , 89 ], ET [ 96 , 97 , 98 , 99 , 100 , 101 ], EOG [ 102 , 103 ], and EMG [ 104 , 105 ].…”

Section: Trends Challenges Prospective Directions and Suggestionsmentioning

confidence: 99%

Brain–Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and Performance

et al. 2021

Brain Sciences

View full text Add to dashboard Cite

The steady-state visual evoked potential (SSVEP), measured by the electroencephalograph (EEG), has high rates of information transfer and signal-to-noise ratio, and has been used to construct brain–computer interface (BCI) spellers. In BCI spellers, the targets of alphanumeric characters are assigned different visual stimuli and the fixation of each target generates a unique SSVEP. Matching the SSVEP to the stimulus allows users to select target letters and numbers. Many BCI spellers that harness the SSVEP have been proposed over the past two decades. Various paradigms of visual stimuli, including the procedure of target selection, layout of targets, stimulus encoding, and the combination with other triggering methods are used and considered to influence on the BCI speller performance significantly. This paper reviews these stimulus paradigms and analyzes factors influencing their performance. The fundamentals of BCI spellers are first briefly described. SSVEP-based BCI spellers, where only the SSVEP is used, are classified by stimulus paradigms and described in chronological order. Furthermore, hybrid spellers that involve the use of the SSVEP are presented in parallel. Factors influencing the performance and visual fatigue of BCI spellers are provided. Finally, prevailing challenges and prospective research directions are discussed to promote the development of BCI spellers.

show abstract

“…Among the various EEG-based BCI paradigms including motor imagery (MI), P300, and steady-state visual-evoked potential (SSVEP) ( Yin et al, 2015 ), SSVEP-based BCIs have been successfully applied to VR applications as a new hands-free communication tool, thanks to its high information transfer rate (ITR) and no (or less) training requirement ( Coogan and He, 2018 ; Stawicki et al, 2018 ; Choi et al, 2019 ; Grichnik et al, 2019 ; Monteiro et al, 2021 ). However, the number of visual stimuli used for SSVEP-based BCIs in VR environment was generally limited to four or five because of the relatively poorer performance of the SSVEP-based BCIs implemented using head-mounted displays (HMDs) than those implemented with conventional liquid crystal displays (LCDs) ( Chen et al, 2015 ; Yao et al, 2018 ). As will be shown in the results of this study, the increment of the number of visual stimuli (e.g., 9) greatly degrades the overall classification accuracy compared to previous studies because the SSVEP-based BCIs implemented with VR-HMDs are more prone to be affected by the peripheral vision due to the relatively short distance between the eyes and the display.…”

Section: Introductionmentioning

confidence: 99%

“…To effectively increase the number of target visual stimuli and/or elevate the overall performance of BCIs, a variety of hybrid BCIs combining EEG with other physiological signals, including video-oculography (VOG) ( Hong and Khan, 2017 ; Stawicki et al, 2017 ; Ma et al, 2018 ; Yao et al, 2018 ) and electrooculogram (EOG) ( Saravanakumar and Ramasubba Reddy, 2019 , 2020 ), have been developed. VOG is a non-invasive, video-based method to measure the movements of the eyes and is known to provide a high-precision estimate of the absolute coordinates of the eye gaze.…”

Section: Introductionmentioning

confidence: 99%

Novel Hybrid Brain-Computer Interface for Virtual Reality Applications Using Steady-State Visual-Evoked Potential-Based Brain–Computer Interface and Electrooculogram-Based Eye Tracking for Increased Information Transfer Rate

Park

2022

Front. Neuroinform.

View full text Add to dashboard Cite

Brain–computer interfaces (BCIs) based on electroencephalogram (EEG) have recently attracted increasing attention in virtual reality (VR) applications as a promising tool for controlling virtual objects or generating commands in a “hands-free” manner. Video-oculography (VOG) has been frequently used as a tool to improve BCI performance by identifying the gaze location on the screen, however, current VOG devices are generally too expensive to be embedded in practical low-cost VR head-mounted display (HMD) systems. In this study, we proposed a novel calibration-free hybrid BCI system combining steady-state visual-evoked potential (SSVEP)-based BCI and electrooculogram (EOG)-based eye tracking to increase the information transfer rate (ITR) of a nine-target SSVEP-based BCI in VR environment. Experiments were repeated on three different frequency configurations of pattern-reversal checkerboard stimuli arranged in a 3 × 3 matrix. When a user was staring at one of the nine visual stimuli, the column containing the target stimulus was first identified based on the user’s horizontal eye movement direction (left, middle, or right) classified using horizontal EOG recorded from a pair of electrodes that can be readily incorporated with any existing VR-HMD systems. Note that the EOG can be recorded using the same amplifier for recording SSVEP, unlike the VOG system. Then, the target visual stimulus was identified among the three visual stimuli vertically arranged in the selected column using the extension of multivariate synchronization index (EMSI) algorithm, one of the widely used SSVEP detection algorithms. In our experiments with 20 participants wearing a commercial VR-HMD system, it was shown that both the accuracy and ITR of the proposed hybrid BCI were significantly increased compared to those of the traditional SSVEP-based BCI in VR environment.

show abstract

High-Speed Spelling in Virtual Reality with Sequential Hybrid BCIs

Cited by 16 publications

References 12 publications

Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs

Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs

Brain–Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and Performance

Novel Hybrid Brain-Computer Interface for Virtual Reality Applications Using Steady-State Visual-Evoked Potential-Based Brain–Computer Interface and Electrooculogram-Based Eye Tracking for Increased Information Transfer Rate

Contact Info

Product

Resources

About