Explanation Perspectives from the Cognitive Sciences---A Survey

Srinivasan, Ramya; Chander, Ajay

doi:10.24963/ijcai.2020/670

Cited by 28 publications

(21 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, this property is about how something is explained instead of what is explained. Examples include the usage of higher-level information [9], abstractions [63,246] or suitable terminology [246], and not using explanations that are circular [238].…”

Section: Co-12mentioning

confidence: 99%

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

Nauta¹,

Trienes²,

Pathak³

et al. 2022

Preprint

View full text Add to dashboard Cite

The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes, also raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practice of more than 300 papers published in the last 7 years at major AI and ML conferences that introduce an XAI method. We find that 1 in 3 papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate with users. We also contribute to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. This systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark and compare new and existing XAI methods. This also opens up opportunities to include quantitative metrics as optimization criteria during model training in order to optimize for accuracy and interpretability simultaneously.

show abstract

Section: Co-12mentioning

confidence: 99%

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

Nauta¹,

Trienes²,

Pathak³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep neural nets lack interpretability as it is difficult to analyze which modalities or features are driving the predictions [26].In real-world scenarios impacting human lives by automated algorithmic decision assistance such as in legal, healthcare, finance, transport, military, and autonomous vehicles, we expect AI systems to provide their predictions with proper evidence and justification [29]. The system's explanation should be human interpretable and understandable, mapping the human mental model to build trust, transparency, reliability for success and failure, robust, fair, and unbiased applications underlying ethical machine learning principles [139], [140]. Explainability is a legal concern to comply with the EU General Data Protection and Regulation (GDPR) act asking for" Right to explanation" to the users of an automated decision-making system [141].…”

Section: Explainable Ai (Xai)mentioning

confidence: 99%

A Review on Explainability in Multimodal Deep Neural Nets

2021

View full text Add to dashboard Cite

Artificial Intelligence techniques powered by deep neural nets have achieved much success in several application domains, most significantly and notably in the Computer Vision applications and Natural Language Processing tasks. Surpassing human-level performance propelled the research in the applications where different modalities amongst language, vision, sensory, text play an important role in accurate predictions and identification. Several multimodal fusion methods employing deep learning models are proposed in the literature. Despite their outstanding performance, the complex, opaque and black-box nature of the deep neural nets limits their social acceptance and usability. This has given rise to the quest for model interpretability and explainability, more so in the complex tasks involving multimodal AI methods. This paper extensively reviews the present literature to present a comprehensive survey and commentary on the explainability in multimodal deep neural nets, especially for the vision and language tasks. Several topics on multimodal AI and its applications for generic domains have been covered in this paper, including the significance, datasets, fundamental building blocks of the methods and techniques, challenges, applications, and future trends in this domain.INDEX TERMS deep multimodal learning, explainable AI, interpretability, survey, trends, vision and language research, XAI.

show abstract

“…This means that the problem is not the complexity of the model or the design of methods such as SHAP, PDP or ALE. The actual problem to be solved by the research community is to determine exactly what questions the model stakeholders can pose in explanatory model analysis and what the desired properties of the answers to these questions are (Miller, 2019;Adebayo et al, 2020;Baniecki and Biecek, 2020;Barredo Arrieta et al, 2020;Sokol and Flach, 2020;Srinivasan and Chander, 2020).…”

Section: Conclusion: You Do Not Explain Without a Contextmentioning

confidence: 99%

Do not explain without context: addressing the blind spot of model explanations

Woźnica,

Pękala,

Baniecki

et al. 2021

Preprint

View full text Add to dashboard Cite

The increasing number of regulations and expectations of predictive machine learning models, such as so called right to explanation, has led to a large number of methods promising greater interpretability. High demand has led to a widespread adoption of XAI techniques like Shapley values, Partial Dependence profiles or permutational variable importance. However, we still do not know enough about their properties and how they manifest in the context in which explanations are created by analysts, reviewed by auditors, and interpreted by various stakeholders. This paper highlights a blind spot which, although critical, is often overlooked when monitoring and auditing machine learning models: the effect of the reference data on the explanation calculation. We discuss that many model explanations depend directly or indirectly on the choice of the referenced data distribution.We showcase examples where small changes in the distribution lead to drastic changes in the explanations, such as a change in trend or, alarmingly, a conclusion. Consequently, we postulate that obtaining robust and useful explanations always requires supporting them with a broader context.

show abstract

Explanation Perspectives from the Cognitive Sciences---A Survey

Cited by 28 publications

References 13 publications

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

A Review on Explainability in Multimodal Deep Neural Nets

Do not explain without context: addressing the blind spot of model explanations

Contact Info

Product

Resources

About