Toward Explainable AI for Regression Models

Letzgus, Simon; Wagner, Patrick; Lederer, Jonas; Samek, Wojciech; Müller, Klaus‐Robert; Montavon, Grégoire

doi:10.48550/arxiv.2112.11407

Cited by 5 publications

(6 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A regression based XAI framework could thus accelerate the development of such techniques, because the reasons why the networks fail to generalize might be better understood for both specific local scale features such as where the Gulf Stream leaves the continental shelf and larger scale processes. In further work, we will benefit from the ongoing recent research developments in XAI for regression, for example, in Letzgus et al (2021), and aim to apply our methodology to this more challenging problem.…”

Section: Discussionmentioning

confidence: 99%

Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics

Clare

Sonnewald

Lguensat

et al. 2022

J Adv Model Earth Syst

View full text Add to dashboard Cite

The trustworthiness of neural networks is often challenged because they lack the ability to express uncertainty and explain their skill. This can be problematic given the increasing use of neural networks in high stakes decision-making such as in climate change applications. We address both issues by successfully implementing a Bayesian Neural Network (BNN), where parameters are distributions rather than deterministic, and applying novel implementations of explainable AI (XAI) techniques. The uncertainty analysis from the BNN provides a comprehensive overview of the prediction more suited to practitioners' needs than predictions from a classical neural network. Using a BNN means we can calculate the entropy (i.e. uncertainty) of the predictions and determine if the probability of an outcome is statistically significant. To enhance trustworthiness, we also spatially apply the two XAI techniques of Layer-wise Relevance Propagation (LRP) and SHapley Additive exPlanation (SHAP) values. These XAI methods reveal the extent to which the BNN is suitable and/or trustworthy. Using two techniques gives a more holistic view of BNN skill and its uncertainty, as LRP considers neural network parameters, whereas SHAP considers changes to outputs. We verify these techniques using comparison with intuition from physical theory. The differences in explanation identify potential areas where new physical theory guided studies are needed.

show abstract

Section: Discussionmentioning

confidence: 99%

Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics

Clare

Sonnewald

Lguensat

et al. 2022

J Adv Model Earth Syst

View full text Add to dashboard Cite

show abstract

“…The most popular explainability techniques were designed for classifiers, and special care needs to be taken when using them for regression, see [74]. Here we adopt the simple yet powerful strategy of [75], where explanations of deep regression models are obtained directly using methods for classification.…”

Section: Methodsmentioning

confidence: 99%

Femtosecond pulse parameter estimation from photoelectron momenta using machine learning

et al. 2023

View full text Add to dashboard Cite

Deep learning models have provided huge interpretation power for image-like data. Specifically, convolutional neural networks (CNNs) have demonstrated incredible acuity for tasks such as feature extraction or parameter estimation. Here we test CNNs on strong-field ionization photoelectron spectra, training on theoretical data sets to `invert' experimental data. Pulse characterization is used as a `testing ground', specifically we retrieve the laser intensity, where `traditional' measurements typically lead to 20% uncertainty. We report on crucial data augmentation techniques required to successfully train on theoretical data and return consistent results from experiments, including accounting for detector saturation. The same procedure can be repeated to apply CNNs in a range of scenarios for strong-field ionization. Using a predictive uncertainty estimation, reliable laser intensity uncertainties of a few percent can be extracted, which are consistently lower than those given by traditional techniques. Using interpretability methods can reveal parts of the distribution that are most sensitive to laser intensity, which can be directly associated with holographic interferences. The CNNs employed provide an accurate and convenient ways to extract parameters, and represent a novel interpretational tool for strong-field ionization spectra.

show abstract

“…A wide range of machine learning (ML) approaches allows for explaining the chemistry of molecules, attributing which parts of the molecules are responsible for the chemical property of interest [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] , and lessening the black box challenge of machine learning 20,21 . Typical explainable ML approaches that provide atomwise attribution include dummy atoms 22 , classification of atoms by chemical intuition 23 , regression models 24 , graph neural network (GNN) attributions [25][26][27][28] with gradients 29 , perturbations 30 , decompositions 31 , and surrogates 32 . In contrast, fragment-based explainability approaches generate importance values for groups of atoms or functional groups (subgraphs), e.g., Hammett equation 33 , matched molecular pairs [34][35][36] , molecular scaffolds 18,37 , and counterfactuals 38 .…”

Section: Background and Summarymentioning

confidence: 99%

“…Various metrics have also been proposed to quantify the accuracy of explanation values. Some metrics focus on characterizing post hoc explanations, such as Grad-Cam 48 , non-zero reference 24 , and the assumption of node explanation smoothness 60 . Other metrics focus on the accuracy of the explanations with respect to the ground truth 41,60 , which relies on the availability of ground-truth explanation datasets.…”

Section: Background and Summarymentioning

confidence: 99%

Ground truth explanation dataset for chemical property prediction on molecular graphs

Hruska

Zhao

Liu

2022

Preprint

View full text Add to dashboard Cite

Interpretation of chemistry on an atomic scale improves with explainable artificial intelligence (XAI). The parts of the molecule with the most significant influence on the chemical property of interest can be visualized with atomwise and bondwise attributions. Nonetheless, the attributions from different XAI methods regularly disagree substantially, causing uncertainty about which explanation is correct. To determine a ground truth for attributions, we define chemical operations which avoid alchemical steps or approximations and allow extracting one attribution per atom or bond from existing datasets of chemical properties. This general procedure allows for generating large datasets of ground truth attributions. The approach allowed us to create a ground truth explanation dataset with more than 5 million data points for the HOMO-LUMO gap chemical property. This open-source dataset of atomistic ground truth explanations may serve as a reference for XAI approaches.

show abstract

Toward Explainable AI for Regression Models

Cited by 5 publications

References 58 publications

Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics

Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics

Femtosecond pulse parameter estimation from photoelectron momenta using machine learning

Ground truth explanation dataset for chemical property prediction on molecular graphs

Contact Info

Product

Resources

About