Explainable artificial intelligence model to predict brain states from fNIRS signals

Abstract

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode’s output onto the input variables for fNIRS signals is described here.Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model’s output in terms of the model’s input.Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations.Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.

Unlocking the neural mechanisms of consumer loan evaluations: an fNIRS and ML-based consumer neuroscience study

“…An overall accuracy rate exceeding 75% was achieved when investigating whether referring to a product as expensive or inexpensive could influence its perceived value-for-money ( Misawa et al, 2014 ). In mental imagery, an accuracy of over 96% was achieved ( Shibu et al, 2023 ). Moreover, these machine learning models can differentiate between mind wandering and task-related episodes with an accuracy of 73.2% ( Liu et al, 2021 ).…”