From Blackbox to Explainable AI in Healthcare: Existing Tools and Case Studies

Abstract

Introduction. Artificial intelligence (AI) models have been employed to automate decision-making, from commerce to more critical fields directly affecting human lives, including healthcare. Although the vast majority of these proposed AI systems are considered black box models that lack explainability, there is an increasing trend of attempting to create medical explainable Artificial Intelligence (XAI) systems using approaches such as attention mechanisms and surrogate models. An AI system is said to be explainable if humans can tell how the system reached its decision. Various XAI-driven healthcare approaches and their performances in the current study are discussed. The toolkits used in local and global post hoc explainability and the multiple techniques for explainability pertaining the Rational, Data, and Performance explainability are discussed in the current study. Methods. The explainability of the artificial intelligence model in the healthcare domain is implemented through the Local Interpretable Model-Agnostic Explanations and Shapley Additive Explanations for better comprehensibility of the internal working mechanism of the original AI models and the correlation among the feature set that influences decision of the model. Results. The current state-of-the-art XAI-based and future technologies through XAI are reported on research findings in various implementation aspects, including research challenges and limitations of existing models. The role of XAI in the healthcare domain ranging from the earlier prediction of future illness to the disease’s smart diagnosis is discussed. The metrics considered in evaluating the model’s explainability are presented, along with various explainability tools. Three case studies about the role of XAI in the healthcare domain with their performances are incorporated for better comprehensibility. Conclusion. The future perspective of XAI in healthcare will assist in obtaining research insight in the healthcare domain.

Explainable AI Over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions

“…Further, the role of XAI in the healthcare sector is gaining importance in conjunction with IoT for disease prediction and diagnosis. The work in [19] deals with the XAI models that enable IoT frameworks used in the medical field to address the challenges involved in the prediction and diagnosis of diseases.…”

Highly Accurate Trimesh and PointNet based algorithm for Gesture and Hindi air writing recognition

“…Classification of the data is one of the most important aspects of gesture recognition. From time to time several methods of classification and optimization have been proposed by researchers [11,12] but PointNet is one of the most popular techniques for point cloud classification and segmentation. PointNet network architecture uses point clouds as input.…”

A novel method to derive personalized minimum viable recommendations for type 2 diabetes prevention based on counterfactual explanations

“…XAI techniques either focus on the representation of the overall behavior of the model, providing global explanations , or describe the decision-making process of the model for a single specific instance providing local explanations [ 8 , 9 ]. Explainability, transparency and trustworthiness are extremely important requirements, especially when it comes to automated solutions that could make life-critical decisions (e.g., clinical decision support systems, medical robotic systems, or autonomous vehicles) and, as such, XAI techniques have been extensively applied to medical case studies in recent years [ 10 – 12 ]. However, transparency of a model is only one of the several components of trustworthiness and even a fully explainable AI system should be compliant with several other requirements in order to be considered trustworthy [ 13 , 14 ].…”