CX-ToM: Counterfactual explanations with theory-of-mind for enhancing human trust in image recognition models

Akula, Arjun R.; Wang, Keze; Liu, Changsong; Saba-Sadiya, Sari; Lu, Hongjing; Todorovic, Sinisa; Chai, Joyce; Zhu, Song‐Chun

doi:10.1016/j.isci.2021.103581

Cited by 37 publications

(13 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the absence of systematic development of XAI, it is a trend to use local interpretation methods to explain the cases studied. In the case of CNNs and their use in medical images, a saliency map is a simple tool for obtaining an explanation of the areas of interest of the network [ 97 ]. Eight interpretable methods of saliency maps (+ Grad-CAM, guided backpropagation, and guided Grad-CAM) were evaluated [ 19 ].…”

Section: Discussion Challenges and Prospectsmentioning

confidence: 99%

Survey of Explainable AI Techniques in Healthcare

Chaddad

Peng

et al. 2023

Sensors

169

View full text Add to dashboard Cite

Artificial intelligence (AI) with deep learning models has been widely applied in numerous domains, including medical imaging and healthcare tasks. In the medical field, any judgment or decision is fraught with risk. A doctor will carefully judge whether a patient is sick before forming a reasonable explanation based on the patient’s symptoms and/or an examination. Therefore, to be a viable and accepted tool, AI needs to mimic human judgment and interpretation skills. Specifically, explainable AI (XAI) aims to explain the information behind the black-box model of deep learning that reveals how the decisions are made. This paper provides a survey of the most recent XAI techniques used in healthcare and related medical imaging applications. We summarize and categorize the XAI types, and highlight the algorithms used to increase interpretability in medical imaging topics. In addition, we focus on the challenging XAI problems in medical applications and provide guidelines to develop better interpretations of deep learning models using XAI concepts in medical image and text analysis. Furthermore, this survey provides future directions to guide developers and researchers for future prospective investigations on clinical topics, particularly on applications with medical imaging.

show abstract

Section: Discussion Challenges and Prospectsmentioning

confidence: 99%

Survey of Explainable AI Techniques in Healthcare

Chaddad

Peng

et al. 2023

Sensors

169

View full text Add to dashboard Cite

show abstract

“…We also note that inferring a user's cognition is currently an active area of research (Akula et al 2022;Shergadwala, Panchal, and Bilionis 2022;Wu et al 2022). While some human-centric requirements may point to the user's desire to be "understood" by a system in real-time, it is currently not practically feasible to so.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

A Human-Centric Perspective on Model Monitoring

Shergadwala¹,

Lakkaraju

Kenthapadi³

2022

HCOMP

View full text Add to dashboard Cite

Predictive models are increasingly used to make various consequential decisions in high-stakes domains such as healthcare, finance, and policy. It becomes critical to ensure that these models make accurate predictions, are robust to shifts in the data, do not rely on spurious features, and do not unduly discriminate against minority groups. To this end, several approaches spanning various areas such as explainability, fairness, and robustness have been proposed in recent literature. Such approaches need to be human-centered as they cater to the understanding of the models to their users. However, there is little to no research on understanding the needs and challenges in monitoring deployed machine learning (ML) models from a human-centric perspective. To address this gap, we conducted semi-structured interviews with 13 practitioners who are experienced with deploying ML models and engaging with customers spanning domains such as financial services, healthcare, hiring, online retail, computational advertising, and conversational assistants. We identified various human-centric challenges and requirements for model monitoring in real-world applications. Specifically, we found that relevant stakeholders would want model monitoring systems to provide clear, unambiguous, and easy-to-understand insights that are readily actionable. Furthermore, our study also revealed that stakeholders desire customization of model monitoring systems to cater to domain-specific use cases.

show abstract

“…R. Soc. B 377: 20210340 explanations has been found to increase understanding and trust in the decisions made by the algorithms [58].…”

Section: (A) Counterfactual Information Plays a Role In Causal Reasoningmentioning

confidence: 99%

“…Counterfactual simulations are used to explain the behaviour of these algorithms in [57] by demonstrating what would have happened if different input values had been entered. The use of these counterfactual explanations has been found to increase understanding and trust in the decisions made by the algorithms [58].…”

Section: Motivationally Relevant Characteristics Of Counterfactual In...mentioning

confidence: 99%

Counterfactual curiosity: motivated thinking about what might have been

FitzGibbon

Murayama

2022

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate ‘counterfactual curiosity’: they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally—the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue ‘Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny’.

show abstract

CX-ToM: Counterfactual explanations with theory-of-mind for enhancing human trust in image recognition models

Abstract: HighlightsAttention is not a Good Explanation Explanation is an Interactive Communication ProcessWe introduce a new XAI framework based on Theory-of-Mind and counterfactual explanations.

Cited by 37 publications

References 78 publications

Survey of Explainable AI Techniques in Healthcare

Survey of Explainable AI Techniques in Healthcare

A Human-Centric Perspective on Model Monitoring

Counterfactual curiosity: motivated thinking about what might have been

Contact Info

Product

Resources

About