A Novel Local Ablation Approach for Explaining Multimodal Classifiers

Miller

Calhoun

2022

Preprint

Self Cite

In recent years, many neuroimaging studies have begun to integrate gradient-based explainability methods to provide insight into key features. However, existing explainability approaches typically generate a point estimate of importance and do not provide insight into the degree of uncertainty associated with explanations. In this study, we present a novel approach for estimating explanation uncertainty for convolutional neural networks (CNN) trained on neuroimaging data. We train a CNN for classification of individuals with schizophrenia (SZs) and controls (HCs) using resting state functional magnetic resonance imaging (rs-fMRI) dynamic functional network connectivity (dFNC) data. We apply Monte Carlo batch normalization (MCBN) and generate an explanation following each iteration using layer-wise relevance propagation (LRP). We then examine whether the resulting distribution of explanations differs between SZs and HCs and examine the relationship between MCBN-based LRP and regular LRP. We find a number of significant differences in LRP relevance for SZs and HCs and find that traditional LRP values frequently diverge from the MCBN relevance distribution. This study provides a novel approach for obtaining insight into the level of uncertainty associated with gradient-based explanations in neuroimaging and represents a significant step towards increasing reliability of explainability methods for a clinical setting.

Section: Introductionmentioning

confidence: 99%

An Approach for Estimating Explanation Uncertainty in fMRI dFNC Classification

Miller

Calhoun

2022

Preprint

Self Cite

“…As a result, most studies have not used explainability ( Zhang et al, 2011 ; Kwon et al, 2018 ; Niroshana et al, 2019 ; Phan et al, 2019 ; Wang et al, 2020 ; Li et al, 2021 ), which is concerning because transparency is increasingly required to assist with model development and physician decision making ( Sullivan and Schweikart, 2019 ). As such, more multimodal explainability methods need to be developed ( Lin et al, 2019 ; Mellem et al, 2020 ; Ellis et al, 2021a , b , c , d ). In this study, we use automated sleep stage classification as a testbed for the development of multimodal explainability methods.…”

Section: Introductionmentioning

confidence: 99%

“…Because of this, they can also be analyzed on a subject-specific level that paves the way for the identification of personalized biomarkers. Furthermore, local explanations can be used to examine the degree to which demographic and clinical variables affect the patterns learned by a classifier for specific classes and features ( Ellis et al, 2021c ), which is a capacity that has not previously been exploited in multimodal classification. Local methods have been applied in a couple multimodal classification studies.…”

Section: Introductionmentioning

confidence: 99%

“…We introduce a global ablation approach that is uniquely adapted for the electrophysiology domain ( Ellis et al, 2021b ). We then present a local ablation approach ( Ellis et al, 2021c ) and show how GBFA methods can be used for local insight into multimodal classifiers ( Ellis et al, 2021a ). With our local methods, we identify subject-level differences in modality importance that support the viability of the methods for personalized biomarker identification.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Novel methods for elucidating modality importance in multimodal electrophysiology classifiers

Sendi

Zhang

et al. 2023

Front. Neuroinform.

Self Cite

IntroductionMultimodal classification is increasingly common in electrophysiology studies. Many studies use deep learning classifiers with raw time-series data, which makes explainability difficult, and has resulted in relatively few studies applying explainability methods. This is concerning because explainability is vital to the development and implementation of clinical classifiers. As such, new multimodal explainability methods are needed.MethodsIn this study, we train a convolutional neural network for automated sleep stage classification with electroencephalogram (EEG), electrooculogram, and electromyogram data. We then present a global explainability approach that is uniquely adapted for electrophysiology analysis and compare it to an existing approach. We present the first two local multimodal explainability approaches. We look for subject-level differences in the local explanations that are obscured by global methods and look for relationships between the explanations and clinical and demographic variables in a novel analysis.ResultsWe find a high level of agreement between methods. We find that EEG is globally the most important modality for most sleep stages and that subject-level differences in importance arise in local explanations that are not captured in global explanations. We further show that sex, followed by medication and age, had significant effects upon the patterns learned by the classifier.DiscussionOur novel methods enhance explainability for the growing field of multimodal electrophysiology classification, provide avenues for the advancement of personalized medicine, yield unique insights into the effects of demographic and clinical variables upon classifiers, and help pave the way for the implementation of multimodal electrophysiology clinical classifiers.

“…While this application can improve model performance, applying traditional explainability methods to raw time-series samples makes it very difficult to know what time or frequency features are extracted by classifiers and to draw global conclusions about the importance of extracted features ( Sturm et al, 2016 ). It should be noted that this difficulty is not applicable to identifying spatial importance ( Sturm et al, 2016 ) or modality importance ( Ellis et al, 2021a , b , f , 2022 ), in multichannel or multimodal classification, respectively. However, this difficultyis applicable when trying to understand the temporal and spectral features extracted by classifiers.…”

Section: Introductionmentioning

confidence: 99%

A Systematic Approach for Explaining Time and Frequency Features Extracted by Convolutional Neural Networks From Raw Electroencephalography Data

Miller

Calhoun

2022

Front. Neuroinform.

In recent years, the use of convolutional neural networks (CNNs) for raw resting-state electroencephalography (EEG) analysis has grown increasingly common. However, relative to earlier machine learning and deep learning methods with manually extracted features, CNNs for raw EEG analysis present unique problems for explainability. As such, a growing group of methods have been developed that provide insight into the spectral features learned by CNNs. However, spectral power is not the only important form of information within EEG, and the capacity to understand the roles of specific multispectral waveforms identified by CNNs could be very helpful. In this study, we present a novel model visualization-based approach that adapts the traditional CNN architecture to increase interpretability and combines that inherent interpretability with a systematic evaluation of the model via a series of novel explainability methods. Our approach evaluates the importance of spectrally distinct first-layer clusters of filters before examining the contributions of identified waveforms and spectra to cluster importance. We evaluate our approach within the context of automated sleep stage classification and find that, for the most part, our explainability results are highly consistent with clinical guidelines. Our approach is the first to systematically evaluate both waveform and spectral feature importance in CNNs trained on resting-state EEG data.