<scp>XAITK</scp>: The explainable AI toolkit

Hu, Brian; Tunison, Paul; Vasu, Bhavan; Menon, Nitesh; Collins, Roderic; Hoogs, Anthony

doi:10.1002/ail2.40

Cited by 18 publications

(5 citation statements)

References 17 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A possible extension could include the utilization of recent state-of-the-art models [41] with more advanced and complex architectures as well as the use of heatmap different saliency algorithms for heatmap calculation. Some interesting works presented by RichardWebster et al [42] and Hu et al [43] used and proposed several algorithms for calculating saliency algorithms. An adoption of the proposed approach to their frameworks could provide useful conclusions from the factual and counterfactual explanations.…”

Section: Discussionmentioning

confidence: 99%

Explainable Image Similarity: Integrating Siamese Networks and Grad-CAM

Livieris,

Pintelas,

Kiriakidou

et al. 2023

J. Imaging

View full text Add to dashboard Cite

With the proliferation of image-based applications in various domains, the need for accurate and interpretable image similarity measures has become increasingly critical. Existing image similarity models often lack transparency, making it challenging to understand the reasons why two images are considered similar. In this paper, we propose the concept of explainable image similarity, where the goal is the development of an approach, which is capable of providing similarity scores along with visual factual and counterfactual explanations. Along this line, we present a new framework, which integrates Siamese Networks and Grad-CAM for providing explainable image similarity and discuss the potential benefits and challenges of adopting this approach. In addition, we provide a comprehensive discussion about factual and counterfactual explanations provided by the proposed framework for assisting decision making. The proposed approach has the potential to enhance the interpretability, trustworthiness and user acceptance of image-based systems in real-world image similarity applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Explainable Image Similarity: Integrating Siamese Networks and Grad-CAM

Livieris,

Pintelas,

Kiriakidou

et al. 2023

J. Imaging

View full text Add to dashboard Cite

show abstract

“…Explainability can be achieved by visualizing the attention maps of the model, which show which parts of the image the model is focusing on when making a prediction. Other methods include saliency maps (Petsiuk et al 2021), which highlight the most important pixels for a prediction, and decision trees, which provide a simple and interpretable representation of the model's decision process (Hu et al 2023). Therefore, few-shot object detection methods have shown promising results in detecting novel objects in aerial images with limited annotated samples.…”

Section: Datasetsmentioning

confidence: 99%

Unlocking the capabilities of explainable few-shot learning in remote sensing

Lee,

Dam,

Ferdaus

et al. 2024

Artif Intell Rev

View full text Add to dashboard Cite

Recent advancements have significantly improved the efficiency and effectiveness of deep learning methods for image-based remote sensing tasks. However, the requirement for large amounts of labeled data can limit the applicability of deep neural networks to existing remote sensing datasets. To overcome this challenge, few-shot learning has emerged as a valuable approach for enabling learning with limited data. While previous research has evaluated the effectiveness of few-shot learning methods on satellite-based datasets, little attention has been paid to exploring the applications of these methods to datasets obtained from Unmanned Aerial Vehicles (UAVs), which are increasingly used in remote sensing studies. In this review, we provide an up-to-date overview of both existing and newly proposed few-shot classification techniques, along with appropriate datasets that are used for both satellite-based and UAV-based data. We demonstrate few-shot learning can effectively handle the diverse perspectives in remote sensing data. As an example application, we evaluate state-of-the-art approaches on a UAV disaster scene dataset, yielding promising results. Furthermore, we highlight the significance of incorporating explainable AI (XAI) techniques into few-shot models. In remote sensing, where decisions based on model predictions can have significant consequences, such as in natural disaster response or environmental monitoring, the transparency provided by XAI is crucial. Techniques like attention maps and prototype analysis can help clarify the decision-making processes of these complex models, enhancing their reliability. We identify key challenges including developing flexible few-shot methods to handle diverse remote sensing data effectively. This review aims to equip researchers with an improved understanding of few-shot learning’s capabilities and limitations in remote sensing, while pointing out open issues to guide progress in efficient, reliable and interpretable data-efficient techniques.

show abstract

“…They found that both SHAP and LIME lacked all three of these qualities, leading to the conclusion that more investigation is needed in the area of explainability. In this context, it is also worth mentioning the explainable AI (XAI) toolkit [10] that is built on top of DARPA's efforts in XAI. The toolkit is purported to be a resource for everyone wanting to use AI responsibly.…”

Section: Introductionmentioning

confidence: 99%

Assessing the Reliability of Machine Learning Models Applied to the Mental Health Domain Using Explainable AI

Pendyala,

Kim

2024

Electronics

View full text Add to dashboard Cite

Machine learning is increasingly and ubiquitously being used in the medical domain. Evaluation metrics like accuracy, precision, and recall may indicate the performance of the models but not necessarily the reliability of their outcomes. This paper assesses the effectiveness of a number of machine learning algorithms applied to an important dataset in the medical domain, specifically, mental health, by employing explainability methodologies. Using multiple machine learning algorithms and model explainability techniques, this work provides insights into the models’ workings to help determine the reliability of the machine learning algorithm predictions. The results are not intuitive. It was found that the models were focusing significantly on less relevant features and, at times, unsound ranking of the features to make the predictions. This paper therefore argues that it is important for research in applied machine learning to provide insights into the explainability of models in addition to other performance metrics like accuracy. This is particularly important for applications in critical domains such as healthcare.

show abstract

XAITK: The explainable AI toolkit

Cited by 18 publications

References 17 publications

Explainable Image Similarity: Integrating Siamese Networks and Grad-CAM

Explainable Image Similarity: Integrating Siamese Networks and Grad-CAM

Unlocking the capabilities of explainable few-shot learning in remote sensing

Assessing the Reliability of Machine Learning Models Applied to the Mental Health Domain Using Explainable AI

Contact Info

Product

Resources

About