Explaining CNN and RNN Using Selective Layer-Wise Relevance Propagation

Jung, Yeon-Jee; Han, Seung-Ho; Choi, Ho-Jin

doi:10.1109/access.2021.3051171

Cited by 31 publications

(25 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experimental results proved that the representations at different layers hold diverse levels of meaning. Jung et al (2021) proposed a novel relevance-based algorithm, which was called elective layer-wise relevance propagation, to explain the image classification and the text classification model?s output through the visualization. The authors used the activations selectively to calculate each activation?s gradient for the output label, which incorporated the true positive gradients during the computational process.…”

Section: • Convolutional Neural Networkmentioning

confidence: 99%

Explainable artificial intelligence: a comprehensive review

et al. 2021

View full text Add to dashboard Cite

Thanks to the exponential growth in computing power and vast amounts of data, artificial intelligence (AI) has witnessed remarkable developments in recent years, enabling it to be ubiquitously adopted in our daily lives. Even though AI-powered systems have brought competitive advantages, the black-box nature makes them lack transparency and prevents them from explaining their decisions. This issue has motivated the introduction of explainable artificial intelligence (XAI), which promotes AI algorithms that can show their internal process and explain how they made decisions. The number of XAI research has increased significantly in recent years, but there lacks a unified and comprehensive review of the latest XAI progress. This review aims to bridge the gap by discovering the critical perspectives of the rapidly growing body of research associated with XAI. After offering the readers a solid XAI background, we analyze and review various XAI methods, which are grouped into (i) pre-modeling explainability, (ii) interpretable model, and (iii) post-modeling explainability. We also pay attention to the current methods that dedicate to interpret and analyze deep learning methods. In addition, we systematically discuss various XAI challenges, such as the trade-off between the performance and the explainability, evaluation methods, security, and policy. Finally, we show the standard approaches that are leveraged to deal with the mentioned challenges.

show abstract

Section: • Convolutional Neural Networkmentioning

confidence: 99%

Explainable artificial intelligence: a comprehensive review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Research has been done over the last years to improve explainabilty on convolutional neural networks by highlighting features on the input image. The approach to overlay heatmaps on input data images has been picked up multiple times [7,11,12] and different algorithms have been described to improve the heatmap's quality. The heatmap is displayed as colored pixels on the original input, where deep red shows areas of interest to the neural network.…”

Section: Related Workmentioning

confidence: 99%

“…The heatmap is displayed as colored pixels on the original input, where deep red shows areas of interest to the neural network. Showing the interest of explainability as a feature for artificial intelligence in general, the approach using heatmaps is adopted onto recurrent neural networks as well [7].…”

Section: Related Workmentioning

confidence: 99%

Towards Explainability of non-Convolutional Neural Networks

Doese

Weis

2021

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

View full text Add to dashboard Cite

Artificial intelligence has risen in popularity in research and applications in the past years. Explainability is a topic that has been proven to work on human interpretable data like images or sentences, but the research is narrow whenever such data is missing.Creating trust through explainability on raw data processing neural networks of any kind will become necessary in the future as networks are evolving further towards artificial general intelligence.This research is focused to visualize parts of the hidden layers instead of focusing explainability on the input data and is independent of the neural network's size. We create a model that represents the neural network in a way that neurons that are activating on similar features are grouped together in structures. This model will be analyzed in a machine-learning equivalent process to identify parts of the network being responsible for a decision.In a further step we use the model to test the processing of raw sensor-data versus an approach of heatmapped explainability with a convolutional neural network. Relevant data points in the input are visualized by a common heatmap approach while the hidden layers are analyzed in this research and should point to structures that have a comparable function in the network. For example, if the heatmap highlights peaks of values, the model will be highlighted in the area that is observed as the activation of the neural network on peaks. We will provide the research of artificial general intelligence with a solution for explainability which is necessary for advanced research and the operation of such applications in complex or dangerous scenarios. CCS CONCEPTS• Artificial intelligence → Explainability; Artificial general intelligence; • Neural networks → Recurrent neural networks; Analysis.

show abstract

“…[20] evaluates how much each type of explanation provides reliable information to people for three different conditions. [21] uses SLRP (a modified version of Layer-wise Relevance Propagation) to find the propagate relevances for each layer of the deep learning models to detect the category of the things in the images, for CNN and RNN. [22] applies Class activation mapping (CAM) to CNN, which uses the weighted sum of image filters of the same size.…”

Section: Introductionmentioning

confidence: 99%

“…In detail, we divide the entire image into many rectangular sub-images with the same width and height. We call each divided sub-image a part, and then we make each modified image by LIME based methods [7,13] feature (or feature masks) [10,11,15,22,25] LRP-based methods [14,16,21] others [9, 12, 17-20, 23, 24] Table 1 compares each paper from the related works by the category of the method used. Among [6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25], [6,8] use the difference of the input values of a neural network to generate explanations.…”

Section: Introductionmentioning

confidence: 99%

An XAI method for convolutional neural networks in self-driving cars

Kim

Joe

2022

PLoS ONE

View full text Add to dashboard Cite

eXplainable Artificial Intelligence (XAI) is a new trend of machine learning. Machine learning models are used to predict or decide something, and they derive output based on a large volume of data set. Here, the problem is that it is hard to know why such prediction was derived, especially when using deep learning models. It makes the models unreliable in the case of reliability-critical applications. So, it is required to explain how they derived such output. It is a reliability-critical application for self-driving cars because the mistakes made by the computers inside them can lead to critical accidents. So, it is necessary to adopt XAI models in this field. In this paper, we propose an XAI method based on computing and explaining the difference of the output values of the neurons in the last hidden layer of convolutional neural networks. First, we input the original image and some modified images of it. Then we derive output values for each image and compare these values. Then, we introduce the Sensitivity Analysis technique to explain which parts of the original image are needed to distinguish the category. In detail, we divide the image into several parts and fill these parts with shades. First, we compute the influence value on the vector indicating the last hidden layer of the model for each of these parts. Then we draw shades whose darkness is in proportion to the influence values. The experimental results show that our approach for XAI in self-driving cars finds the parts needed to distinguish the category of these images accurately.

show abstract

Explaining CNN and RNN Using Selective Layer-Wise Relevance Propagation

Cited by 31 publications

References 15 publications

Explainable artificial intelligence: a comprehensive review

Explainable artificial intelligence: a comprehensive review

Towards Explainability of non-Convolutional Neural Networks

An XAI method for convolutional neural networks in self-driving cars

Contact Info

Product

Resources

About