Recursive Division of Image for Explanation of Shallow CNN Models

Gorokhovatskyi, Oleksii; Peredrii, Olena

doi:10.1007/978-3-030-68796-0_20

Cited by 3 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the problem with this random perturbation is that there will be many additional regions that are also significantly represented. Gorokhovatskyi et al [22] propose a method to explain the hidden parts of CNN, which segments images into w × h square areas and the square areas are occluded by turning into white or black, different areas are occluded sequentially to determine the region in which themodel is interested. Sililarly, the square area division ignores the contour information of elements in the image and does note hava advantage on visual interpretation.…”

Section: Related Workmentioning

confidence: 99%

“…For example, M. D. Zeiler et al [14] use a gray pixel block to occlude a certain part of the image; D. SEO et al [20] process the image by eliminating a portion of super-pixels. O. Gorokhovatskyi et al [22] use black or white square to achieve perturbation. Even though the above methods can help to obtain new required image, they cannot finalize the goal of observing a certain feature in supporting the classification results.…”

Section: Perturbationmentioning

confidence: 99%

See 1 more Smart Citation

Feature-Based Interpretation of Image Classification With the Use of Convolutional Neural Networks

Wang,

Xia,

2024

IEEE Access

View full text Add to dashboard Cite

Deep neural networks have made remarkable contributions in many fields such as computer vision and natural language processing. However, due to their complex internal structures, for a long time, there is a lack of trust in deep neural networks, particularly in the domains like healthcare, finance, traffic safety, etc. To explain the internal mechanism of deep neural networks, a series of interpretable models, which mainly focus on enhancing the interpretability of deep learning models, has been proposed. This study pursues to data interpretability by analyzing the significance of image features with the goal of mining essential information contained in various visual features. Super-pixel segmentation is firstly applied to divide the image into several coherent visual feature regions, and these regions are subsequently overwritten by certain marks. Next, image occlusion is realized by modifying the pixel values of these feature regions according to the marks. Finally, we conduct comparative analytics of the classification results between the original data and occluded data to scrutinize the importance of key visual features. A collection of experimental studies is conducted and the results consistently demonstrate that the proposed method enables to observe the classifier's dependence on different features, offering a new perspective on interpretability rooted in data analytics.INDEX TERMS deep neural networks, model interpretability, image feature analysis, computer vision.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Perturbationmentioning

confidence: 99%

Feature-Based Interpretation of Image Classification With the Use of Convolutional Neural Networks

Wang,

Xia,

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The explanations are maps of the input images assigning different colours to the various types of labels. Similarly, a recursive division method proposed by [184] hides rectangular parts of the input images of varying size to analyse their influence on the predictions of the underlying neural network.…”

Section: Visual Explanations-miscellaneousmentioning

confidence: 99%

Classification of Explainable Artificial Intelligence Methods through Their Output Formats

Vilone

Longo

2021

MAKE

View full text Add to dashboard Cite

Machine and deep learning have proven their utility to generate data-driven models with high accuracy and precision. However, their non-linear, complex structures are often difficult to interpret. Consequently, many scholars have developed a plethora of methods to explain their functioning and the logic of their inferences. This systematic review aimed to organise these methods into a hierarchical classification system that builds upon and extends existing taxonomies by adding a significant dimension—the output formats. The reviewed scientific papers were retrieved by conducting an initial search on Google Scholar with the keywords “explainable artificial intelligence”; “explainable machine learning”; and “interpretable machine learning”. A subsequent iterative search was carried out by checking the bibliography of these articles. The addition of the dimension of the explanation format makes the proposed classification system a practical tool for scholars, supporting them to select the most suitable type of explanation format for the problem at hand. Given the wide variety of challenges faced by researchers, the existing XAI methods provide several solutions to meet the requirements that differ considerably between the users, problems and application fields of artificial intelligence (AI). The task of identifying the most appropriate explanation can be daunting, thus the need for a classification system that helps with the selection of methods. This work concludes by critically identifying the limitations of the formats of explanations and by providing recommendations and possible future research directions on how to build a more generally applicable XAI method. Future work should be flexible enough to meet the many requirements posed by the widespread use of AI in several fields, and the new regulations.

show abstract

“…[39][40][41][42][43] Without such model explainability, deep learning algorithms remain a "black box" in implementation. The massive data computations in deep neural networks are beyond human logical and symbolic abilities for causality, 44 which raises technical issues of deep learning model development for medical imaging applications. These issues include, but are not limited to, the utilization of model input ("Do we need this as a part of the model?…”

Section: Introductionmentioning

confidence: 99%

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi‐parametric MRI‐based glioma segmentation

Yang

et al. 2023

Medical Physics

View full text Add to dashboard Cite

PurposeTo develop a neural ordinary differential equation (ODE) model for visualizing deep neural network behavior during multi‐parametric MRI‐based glioma segmentation as a method to enhance deep learning explainability.MethodsBy hypothesizing that deep feature extraction can be modeled as a spatiotemporally continuous process, we implemented a novel deep learning model, Neural ODE, in which deep feature extraction was governed by an ODE parameterized by a neural network. The dynamics of (1) MR images after interactions with the deep neural network and (2) segmentation formation can thus be visualized after solving the ODE. An accumulative contribution curve (ACC) was designed to quantitatively evaluate each MR image's utilization by the deep neural network toward the final segmentation results.The proposed Neural ODE model was demonstrated using 369 glioma patients with a 4‐modality multi‐parametric MRI protocol: T1, contrast‐enhanced T1 (T1‐Ce), T2, and FLAIR. Three Neural ODE models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The key MRI modalities with significant utilization by deep neural networks were identified based on ACC analysis. Segmentation results by deep neural networks using only the key MRI modalities were compared to those using all four MRI modalities in terms of Dice coefficient, accuracy, sensitivity, and specificity.ResultsAll Neural ODE models successfully illustrated image dynamics as expected. ACC analysis identified T1‐Ce as the only key modality in ET and TC segmentations, while both FLAIR and T2 were key modalities in WT segmentation. Compared to the U‐Net results using all four MRI modalities, the Dice coefficient of ET (0.784→0.775), TC (0.760→0.758), and WT (0.841→0.837) using the key modalities only had minimal differences without significance. Accuracy, sensitivity, and specificity results demonstrated the same patterns.ConclusionThe Neural ODE model offers a new tool for optimizing the deep learning model inputs with enhanced explainability. The presented methodology can be generalized to other medical image‐related deep‐learning applications.

show abstract

Recursive Division of Image for Explanation of Shallow CNN Models

Cited by 3 publications

References 16 publications

Feature-Based Interpretation of Image Classification With the Use of Convolutional Neural Networks

Feature-Based Interpretation of Image Classification With the Use of Convolutional Neural Networks

Classification of Explainable Artificial Intelligence Methods through Their Output Formats

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi‐parametric MRI‐based glioma segmentation

Contact Info

Product

Resources

About