Impact of Feedback Type on Explanatory Interactive Learning

Hagos, Misgina Tsighe; Curran, Kathleen M.; Namee, Brian Mac

doi:10.1007/978-3-031-16564-1_13

Cited by 5 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some Concept Bottleneck Models incorporate two tiers of human knowledge in their pipelines -human-defined concepts used in training; and direct human involvement in rectifying faulty concept predictions during inference. Another popular class of human-in-the-loop interpretability methods [128], [129], [130], [131]), known as eXplainatory Interractive Learning (XIL), employs human supervision to manually edit the heatmaps generated by conventional attribution methods like LIME, CAM Grad-CAM. Although Active Learning (AL) [132] also leverages human in the machine learning loop to improve performance, the fundamental difference is that XIL particularly focuses on achieving this goal by manipulating explanations.…”

Section: Knowledge-informed Explainability Methodsmentioning

confidence: 99%

“…On the other hand, some approaches may enhance their main objective while compromising other figures of merit. The interpretability-accuracy trade-off -which still a subject of intense debate anyway (see [130], [131], [175], [176], [177])-associated with some knowledge-informed techniques is a classic example of this situation.…”

Section: Summary Of the Main Features Of Prior Knowledgeinformed Appr...mentioning

confidence: 99%

See 1 more Smart Citation

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Mumuni

2022

Prog Artif Intell

View full text Add to dashboard Cite

We review current and emerging knowledge-informed and brain-inspired cognitive systems for realizing adversarial defenses, eXplainable Artificial Intelligence (XAI), and zero-shot or few-shot learning. Data-driven deep learning models have achieved remarkable performance and demonstrated capabilities surpassing human experts in many applications. Yet, their inability to exploit domain knowledge leads to serious performance limitations in practical applications. In particular, deep learning systems are exposed to adversarial attacks, which can trick them into making glaringly incorrect decisions. Moreover, complex data-driven models typically lack interpretability or explainability, i.e., their decisions cannot be understood by human subjects. Furthermore, models are usually trained on standard datasets with a closed-world assumption. Hence, they struggle to generalize to unseen cases during inference in practical open-world environments, thus, raising the zero-or few-shot generalization problem. Although many conventional solutions exist, explicit domain knowledge, brain-inspired neural network and cognitive architectures offer powerful new dimensions towards alleviating these problems. Prior knowledge is represented in appropriate forms and incorporated in deep learning frameworks to improve performance. Brain-inspired cognition methods use computational models that mimic the human mind to enhance intelligent behavior in artificial agents and autonomous robots. Ultimately, these models achieve better explainability, higher adversarial robustness and data-efficient learning, and can, in turn, provide insights for cognitive science and neurosciencethat is, to deepen human understanding on how the brain works in general, and how it handles these problems.Index Terms-Domain knowledge, cognitive architecture, brain-inspired neural network, explainable AI, adversarial attack, zero-shot generalization. ✦ INTRODUCTION State of deep learningMachine learning is an artificial intelligence (AI) concept that enables computing systems to learn useful relationships from data and then use this information to identify learned patterns and make predictions on new inputs. Deep learning (DL) is a machine learning method that uses a multi-layered arrangement of computational units to learn relevant patterns from data. Many deep learning constructs have been proposed for various use-cases. Some of the most popular ones are deep convolutional neural networks (DCNN) [1], [2] recurrent neural networks (RNN) [3], generative adversarial networks (GANs) [4], deep reinforcement learning [5] and vision transformers [6], [7]). DCNNs are a class of deep neural networks (DNNs) specially designed to handle image data.Recently, deep learning methods have accomplished remarkable milestones in many critical applications. In medical domains, some state-of-the-art models (e.g., as reported

show abstract

Section: Knowledge-informed Explainability Methodsmentioning

confidence: 99%

Section: Summary Of the Main Features Of Prior Knowledgeinformed Appr...mentioning

confidence: 99%

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Mumuni

2022

Prog Artif Intell

View full text Add to dashboard Cite

show abstract

“…Recent work on explanatory supervision (XS) has shown that models can be successfully protected from learning spurious signals by eliciting ground-truth explanations from humans as an additional supervision for the models (Ross, Hughes, and Doshi-Velez 2017;Teso and Kersting 2019;Rieger et al 2020;Schramowski et al 2020;Hagos, Curran, and Mac Namee 2022;Friedrich et al 2023). Of these methods, "right for the right reasons" (RRR) (Ross, Hughes, and Doshi-Velez 2017) and "right for better reasons" (RBR) (Shao et al 2021) seem to perform particularly well (Friedrich et al 2023).…”

Section: Introductionmentioning

confidence: 99%

Targeted Activation Penalties Help CNNs Ignore Spurious Signals

Zhang,

Williams,

Toni

2024

AAAI

View full text Add to dashboard Cite

Neural networks (NNs) can learn to rely on spurious signals in the training data, leading to poor generalisation. Recent methods tackle this problem by training NNs with additional ground-truth annotations of such signals. These methods may, however, let spurious signals re-emerge in deep convolutional NNs (CNNs). We propose Targeted Activation Penalty (TAP), a new method tackling the same problem by penalising activations to control the re-emergence of spurious signals in deep CNNs, while also lowering training times and memory usage. In addition, ground-truth annotations can be expensive to obtain. We show that TAP still works well with annotations generated by pre-trained models as effective substitutes of ground-truth annotations. We demonstrate the power of TAP against two state-of-the-art baselines on the MNIST benchmark and on two clinical image datasets, using four different CNN architectures.

show abstract

“…An approach that builds on a combination of concept bottleneck models trained on basic concepts like shapes, patterns, colours, et cetera, together with explainable interactive learning (XIL) Hagos et al, 2022), is a promising path forward. In addition, interesting upcoming work by, for example, Mutahar and Miller (2022) builds on a combination of inherently explainable models, such as decision trees and neural networks.…”

Section: Cathy O'neilmentioning

confidence: 99%

NEURAL NETWORKS IN CONTEXT: CHALLENGES AND OPPORTUNITIES A Critical Inquiry into Prerequisites for User Trust in Decisions Promoted by Neural Networks

Holmberg¹

View full text Add to dashboard Cite

Artificial intelligence and machine learning (ML) in particular increasingly impact human life by creating value from collected data. This assetisation affects all aspectsof human life, from choosing a significant other to recommending a product for us to consume. This type of ML-based system thrives because it predicts human behaviour based on average case performance metrics (like accuracy). However, its usefulnessis more limited when it comes to being transparent about its internal knowledge representations for singular decisions, for example, it is not good at explaining why ithas suggested a particular decision in a specific context.The goal of this work is to let end users be in command of how ML systems are used and thereby combine the strengths of humans and machines – machines which can propose transparent decisions. Artificial neural networks are an interesting candidate for a setting of this type, given that this technology has been successful in building knowledge representations from raw data. A neural network can be trained by exposing it to data from the target domain. It can then internalise knowledge representations from the domain and perform contextual tasks. In these situations, the fragment of the actual world internalised in an ML system has to be contextualised by a human to beuseful and trustworthy in non-static settings.This setting is explored through the overarching research question: What challenges and opportunities can emerge when an end user uses neural networks in context to support singular decision-making? To address this question, Research through Design is used as the central methodology, as this research approach matches the openness of the research question. Through six design experiments, I explore and expand on challenges and opportunities in settings where singular contextual decisions matter. The initial design experiments focus on opportunities in settings that augment human cognitive abilities. Thereafter, the experiments explore challenges related to settings where neural networks can enhance human cognitive abilities. This part concerns approaches intended to explain promoted decisions.This work contributes in three ways: 1) exploring learning related to neural networks in context to put forward a core terminology for contextual decision-making using ML systems, wherein the terminology includes the generative notions of true-to-the-domain, concept, out-of-distribution and generalisation; 2) presenting a number of design guidelines; and 3) showing the need to align internal knowledge representations with concepts if neural networks are to produce explainable decisions. I also argue that training neural networks to generalise basic concepts like shapes and colours, concepts easily understandable by humans, is a path forward. This research direction leads towards neural network-based systems that can produce more complex explanations that build on basic generalisable concepts.

show abstract

Impact of Feedback Type on Explanatory Interactive Learning

Cited by 5 publications

References 37 publications

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Targeted Activation Penalties Help CNNs Ignore Spurious Signals

NEURAL NETWORKS IN CONTEXT: CHALLENGES AND OPPORTUNITIES A Critical Inquiry into Prerequisites for User Trust in Decisions Promoted by Neural Networks

Contact Info

Product

Resources

About