Explainable AI and interpretable computer vision : from oversight to insight

Abstract

The last decades have seen rapid development of advanced machine learning (ML) models, considered a subarea of Artificial Intelligence (AI). Massive computing power and growing data availability have allowed the training of deep artificial neural networks, which learn themselves by finding task-relevant patterns in data. The size and complexity of these deep learning models grew over the years in pursuit of predictive performance. However, such black boxes prevent users from assessing whether the learned behaviour is in line with expectations and intentions. We argue in this thesis that the sole focus on predictive performance is an unsustainable trajectory, as a model can make right predictions for the wrong reasons. The research field of Explainable AI (XAI) addresses the black-box nature of ML models by generating explanations that present (aspects of) a model's behaviour in human-understandable terms. This thesis shows that explainability can be used to open up the whole machine learning pipeline: from understanding the underlying data, to assessing the quality of explanations and designing models that are interpretable by design. As such, we support the transition from oversight to insight. in-model uitlegbaarheid, zoals onze part-prototype modellen uit deel II, kunnen modellen worden 'heropgevoed' met onze gewenste normen, waarden en redeneringen. Op die manier kan kunstmatige intelligentie worden aangepast en aangevuld met menselijke intelligentie. Mensen in staat te stellen om ongewenst gedrag van een model te detecteren en te corrigeren draagt bij aan een effectief maar ook betrouwbaar en verantwoord gebruik van AI. v 12 Discussion and Reflection 12.

Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond

“…However, it is crucial to ensure that these systems support human users and operators who bear the final responsibility for decisionmaking. The models should also act as "super-assistants," providing an additional layer of insight and analysis to aid healthcare professionals in their decision-making process [8]. Yet with benefits, limitations of AI can not be ignored [9].…”

Large Language Models: Their Success and Impact

“…While a certain progress has been achieved in this endeavor, substantial work remains to be done in order to successfully dismantle the "black box" phenomenon. This would enable the attainment of explicability, a critical aspect in comprehending and justifying the responses provided by AI systems, as explored by [14] in their work on understanding AI reasoning.…”