Understanding the Role of Self-Attention in a Transformer Model for the Discrimination of SCD From MCI Using Resting-State EEG

Abstract

The identification of EEG biomarkers to discriminate Subjective Cognitive Decline (SCD) from Mild Cognitive Impairment (MCI) conditions is a complex task which requires great clinical effort and expertise. We exploit the self-attention component of the Transformer architecture to obtain physiological explanations of the model's decisions in the discrimination of 56 SCD and 45 MCI patients using resting-state EEG. Specifically, an interpretability workflow leveraging attention scores and time-frequency analysis of EEG epochs through Continuous Wavelet Transform is proposed. In the classification framework, models are trained and validated with 5-fold cross-validation and evaluated on a test set obtained by selecting 20% of the total subjects. Ablation studies and hyperparameter tuning tests are conducted to identify the optimal model configuration. Results show that the best performing model, which achieves acceptable results both on epochs' and patients' classification, is capable of finding specific EEG patterns that highlight changes in the brain activity between the two conditions. We demonstrate the potential of attention weights as tools to guide experts in understanding which disease-relevant EEG features could be discriminative of SCD and MCI.

Prognosticating the outcome of intensive care in older patients—a narrative review

“…New and more sophisticated types of neural networks (transformers) were developed by enhancing the interconnections between their sub-units to detect more complex patterns in data [72,73]. This technology is evolving rapidly, and we would like to refer the interested reader to the latest literature in this domain.…”

Artificial Intelligence in Newborn Medicine

“…Supervised learning techniques are used in medical image analysis, disease classification, and predictive modeling. 59 Transformer models are used in medical NLP tasks, including language translation, text summarization, and clinical documentation. Later in this review, we have focused on GPT and its importance in medicine and its branches such as neonatology.…”