Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs

Abstract

Objective. Latest target recognition methods that are equipped with learning from the subject’s calibration data, represented by the extended canonical correlation analysis (eCCA) and the ensemble task-related component analysis (eTRCA), can achieve extra high performance in the steady-state visual evoked potential (SSVEP)-based brain–computer interfaces (BCIs), however their performance deteriorate drastically if the calibration trials are insufficient. This paper develops a new scheme to learn from limited calibration data. Approach. A learning across multiple stimuli scheme is proposed for the target recognition methods, which applies to learning the data corresponding to not only the target stimulus but also the other stimuli. The resulting optimization problems can be simplified and solved utilizing the prior knowledge and properties of SSVEPs across different stimuli. With the new learning scheme, the eCCA and the eTRCA can be extended to the multi-stimulus eCCA (ms-eCCA) and the multi-stimulus eTRCA (ms-eTRCA), respectively, as well as a combination of them (i.e. ms-eCCA+ms-eTRCA) that incorporates their merits. Main results. Evaluation and comparison using an SSVEP-BCI benchmark dataset with 35 subjects show that the ms-eCCA (or ms-eTRCA) performs significantly better than the eCCA (or eTRCA) method while the ms-eCCA+ms-eTRCA performs the best. With the learning across stimuli scheme, the existing target recognition methods can be further improved in terms of the target recognition performance and the ability against insufficient calibration. Significance. A new learning scheme is proposed towards the efficient use of the calibration data, providing enhanced performance and saving calibration time in the SSVEP-based BCIs.

BETA: A Large Benchmark Database Toward SSVEP-BCI Application

“…Based on whether a calibration or training phase is required for the extraction of spatial filters, the signal detection methods can be categorized into supervised methods and training-free methods. The supervised methods exploit an optimal spatial filter by a training procedure and achieve the state-of-the-art classification performance in the SSVEP-based BCI (Nakanishi et al, 2018;Wong et al, 2020a). These spatial filters or projection direction can be learned by exploiting individual training template (Bin et al, 2011), reference signal optimization (Zhang et al, 2013), interfrequency variation (Yin et al, 2015), and ensemble reference signals (Nakanishi et al, 2014;Chen et al, 2015a) in the framework of canonical correlation analysis (CCA).…”

“…These spatial filters or projection direction can be learned by exploiting individual training template (Bin et al, 2011), reference signal optimization (Zhang et al, 2013), interfrequency variation (Yin et al, 2015), and ensemble reference signals (Nakanishi et al, 2014;Chen et al, 2015a) in the framework of canonical correlation analysis (CCA). Recently, the task-related components (Nakanishi et al, 2018) and the multiple neighboring stimuli (Wong et al, 2020a) have been utilized to derive spatial filters in order to boost the discriminative power of the learned model further. On the other hand, the trainingfree methods perform feature extraction and classification in one step without the training session in the online BCI.…”

Simultaneous Decoding of Eccentricity and Direction Information for a Single-Flicker SSVEP BCI

“…As reported in [29], the signal quality of the dataset will influence the estimate of covariance matrices in the CCA-based methods. Therefore, due to the relatively weak SSVEP responses in the proposed paradigm, the obtained spatial filters in the present study may be not as effective as those from traditional SSVEP studies [30], where the stimuli were directly attended. Consequently, a relatively longer time is needed to make a reliable classification.…”

“…First of all, by constructing spatial filters to make the neural patterns evoked by stimulus at different locations more distinguishable with methods like common spatial patterns [31] and DCPM [32], it is possible to enhance the recognition performance in the proposed paradigm. Furthermore, as an increasing training sample size is expected to boost the classification accuracy in CCA-based methods [30], using more training trials for each direction or exploiting of the training data from other subjects may also improve the average accuracy.…”

Optimizing SSVEP-Based BCI System towards Practical High-Speed Spelling

“…Here, indicates the number of channels, is the number of sampling points and is the spatial filter vector. The spatial filter generated by TRCA has shown excellent performance in recent SSVEP-based BCI systems [ 25 , 30 , 33 ]. For frequency i , the TRCA aims to maximize the reproducibility from trial to trial: where h indicates the index of training trials, and is the number of training trials.…”