Feature Extraction and Classification Methods for Hybrid fNIRS-EEG Brain-Computer Interfaces

Abstract

In this study, a brain-computer interface (BCI) framework for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for locked-in syndrome (LIS) patients is investigated. Brain tasks, channel selection methods, and feature extraction and classification algorithms available in the literature are reviewed. First, we categorize various types of patients with cognitive and motor impairments to assess the suitability of BCI for each of them. The prefrontal cortex is identified as a suitable brain region for imaging. Second, the brain activity that contributes to the generation of hemodynamic signals is reviewed. Mental arithmetic and word formation tasks are found to be suitable for use with LIS patients. Third, since a specific targeted brain region is needed for BCI, methods for determining the region of interest are reviewed. The combination of a bundled-optode configuration and threshold-integrated vector phase analysis turns out to be a promising solution. Fourth, the usable fNIRS features and EEG features are reviewed. For hybrid BCI, a combination of the signal peak and mean fNIRS signals and the highest band powers of EEG signals is promising. For classification, linear discriminant analysis has been most widely used. However, further research on vector phase analysis as a classifier for multiple commands is desirable. Overall, proper brain region identification and proper selection of features will improve classification accuracy. In conclusion, five future research issues are identified, and a new BCI scheme, including brain therapy for LIS patients and using the framework of hybrid fNIRS-EEG BCI, is provided.

Using the General Linear Model to Improve Performance in fNIRS Single Trial Analysis and Classification: A Perspective

“…The majority of fNIRS-based BCI work performed so far relies on the first approach i.e., preprocessing steps such as channel pruning, removal of physiological noise, de-trending and motion artifact removal/correction are applied to the data prior to HRF estimation. The remaining signal is then assumed to represent the estimated hemodynamic brain response and features are extracted/selected from this signal for classification (Matthews et al, 2008;Hong et al, 2018). We summarize below the most commonly used preprocessing steps currently used in the fNIRSbased BCI field.…”

“…In our representative fNIRS-based BCI study sample, the most commonly applied methods for the removal of physiological nuisance signals aside from band-pass filtering (see Figure 3) are ICA (Comon, 1994), EMD (Huang et al, 1998), Transfer Function (TF) models (Pfurtscheller and Florian, 1997), Common Average Reference (CAR) (Pfurtscheller et al, 2010), CWT (Mallat, 1999), and Moving Average Convergence Divergence (MACD) (Appel, 2005). See Matthews et al (2008), Scholkmann et al (2014), Hong and Khan (2017), and Hong et al (2018) for additional methods not mentioned here. ICA is a blind source separation method that assumes statistical independence between non-Gaussian components.…”

“…In noninvasive BCI, EEG is currently the primary modality used for both active and passive domains (Birbaumer et al, 1999;Blankertz et al, 2002Blankertz et al, , 2011Wolpaw et al, 2002;Dornhege, 2007;Müller et al, 2008;Tomioka and Müller, 2010;Zander and Kothe, 2011;Van Erp et al, 2012;Haufe et al, 2014). However, more recently, an increasing number of studies have explored the suitability of functional Near-Infrared Spectroscopy for BCI and single trial classification of evoked brain activity (Matthews et al, 2008;Hong et al, 2018). fNIRS is a non-invasive, non-hazardous optical imaging technique that measures hemodynamic changes associated with brain metabolism (Villringer and Chance, 1997;Ferrari and Quaresima, 2012;Boas et al, 2014).…”

“…Due to its dominance in the field, best practice preprocessing recipes exist for EEG to optimize BCI performance (Parra et al, 2005;Blankertz et al, 2008Blankertz et al, , 2011Müller et al, 2008), but so far not for fNIRS. Along with the growing number of publications that have studied fNIRS-based BCI and single trial analysis over the course of the last several years (see for instance Naseer and Hong, 2015 for a review), a plethora of methods for optimal feature extraction and classification have been investigated (Matthews et al, 2008;Hong and Khan, 2017;Hong et al, 2018). Remarkably, however, well-established methodology from conventional fMRI and fNIRS neuroscience has so far rarely been adopted for fNIRS single trial signal preprocessing i.e., for removing systemic and non-systemic confounding factors from the signal (see Figure 1).…”

Reduction of Onset Delay in Functional Near-Infrared Spectroscopy: Prediction of HbO/HbR Signals

“…Thus far, the features frequently used from these windows include signal mean, signal slope, signal peak, skewness, kurtosis, variance, standard deviation, number and sum of peaks, root mean square, median, etc. (Hwang et al, 2016;Naseer et al, 2016;Liu and Hong, 2017;Hong et al, 2018b;Wibowo et al, 2018).…”