Generating Explanations for Conceptual Validation of Graph Neural Networks: An Investigation of Symbolic Predicates Learned on Relevance-Ranked Sub-Graphs

Finzel, Bettina; Saranti, Anna; Angerschmid, Alessa; Tafler, David; Pfeifer, Bastian

doi:10.1007/s13218-022-00781-7

Cited by 12 publications

(4 citation statements)

References 41 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Knowledge-driven solutions based on logical reasoning over symbolic rules help overcome both of these limitations. By enforcing compliance of explanations with prior knowledge about the tasks, rulebased methods [108], [145] ensure that only sound associations between predictions and explanations are allowed. Also, logic rules are unambiguous and are easy to understand by humans.…”

Section: Knowledge-informed Explainability Methodsmentioning

confidence: 99%

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Mumuni

2022

Prog Artif Intell

View full text Add to dashboard Cite

We review current and emerging knowledge-informed and brain-inspired cognitive systems for realizing adversarial defenses, eXplainable Artificial Intelligence (XAI), and zero-shot or few-shot learning. Data-driven deep learning models have achieved remarkable performance and demonstrated capabilities surpassing human experts in many applications. Yet, their inability to exploit domain knowledge leads to serious performance limitations in practical applications. In particular, deep learning systems are exposed to adversarial attacks, which can trick them into making glaringly incorrect decisions. Moreover, complex data-driven models typically lack interpretability or explainability, i.e., their decisions cannot be understood by human subjects. Furthermore, models are usually trained on standard datasets with a closed-world assumption. Hence, they struggle to generalize to unseen cases during inference in practical open-world environments, thus, raising the zero-or few-shot generalization problem. Although many conventional solutions exist, explicit domain knowledge, brain-inspired neural network and cognitive architectures offer powerful new dimensions towards alleviating these problems. Prior knowledge is represented in appropriate forms and incorporated in deep learning frameworks to improve performance. Brain-inspired cognition methods use computational models that mimic the human mind to enhance intelligent behavior in artificial agents and autonomous robots. Ultimately, these models achieve better explainability, higher adversarial robustness and data-efficient learning, and can, in turn, provide insights for cognitive science and neurosciencethat is, to deepen human understanding on how the brain works in general, and how it handles these problems.Index Terms-Domain knowledge, cognitive architecture, brain-inspired neural network, explainable AI, adversarial attack, zero-shot generalization. ✦ INTRODUCTION State of deep learningMachine learning is an artificial intelligence (AI) concept that enables computing systems to learn useful relationships from data and then use this information to identify learned patterns and make predictions on new inputs. Deep learning (DL) is a machine learning method that uses a multi-layered arrangement of computational units to learn relevant patterns from data. Many deep learning constructs have been proposed for various use-cases. Some of the most popular ones are deep convolutional neural networks (DCNN) [1], [2] recurrent neural networks (RNN) [3], generative adversarial networks (GANs) [4], deep reinforcement learning [5] and vision transformers [6], [7]). DCNNs are a class of deep neural networks (DNNs) specially designed to handle image data.Recently, deep learning methods have accomplished remarkable milestones in many critical applications. In medical domains, some state-of-the-art models (e.g., as reported

show abstract

Section: Knowledge-informed Explainability Methodsmentioning

confidence: 99%

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Mumuni

2022

Prog Artif Intell

View full text Add to dashboard Cite

show abstract

“…Each of them sheds light on a different aspect of the AI model’s computation and many times it has been shown that there is no mutual consent between them, leading to the so-called ‘disagreement’ problem ( Krishna et al., 2022 ). Currently, quality metrics for xAI methods ( Doumard et al., 2023 ; Schwalbe and Finzel, 2023 ) and benchmarks for its evaluation are being defined ( Agarwal et al., 2023 ) to motivate xAI research in directions that support trustworthy, reliable, actionable and causal explanations even if they don’t always align with human pre-conceived notions and expectations ( Holzinger et al., 2019 ; Magister et al., 2021 ; Finzel et al., 2022 ; Saranti et al., 2022 ; Cabitza et al., 2023 ; Holzinger et al., 2023c ).…”

Section: Accelerating Plant Breeding Processes With Explainable Aimentioning

confidence: 99%

Reviewing the essential roles of remote phenotyping, GWAS and explainable AI in practical marker-assisted selection for drought-tolerant winter wheat breeding

Chang-Brahim,

Koppensteiner,

Beltrame

et al. 2024

Front. Plant Sci.

Self Cite

View full text Add to dashboard Cite

Marker-assisted selection (MAS) plays a crucial role in crop breeding improving the speed and precision of conventional breeding programmes by quickly and reliably identifying and selecting plants with desired traits. However, the efficacy of MAS depends on several prerequisites, with precise phenotyping being a key aspect of any plant breeding programme. Recent advancements in high-throughput remote phenotyping, facilitated by unmanned aerial vehicles coupled to machine learning, offer a non-destructive and efficient alternative to traditional, time-consuming, and labour-intensive methods. Furthermore, MAS relies on knowledge of marker-trait associations, commonly obtained through genome-wide association studies (GWAS), to understand complex traits such as drought tolerance, including yield components and phenology. However, GWAS has limitations that artificial intelligence (AI) has been shown to partially overcome. Additionally, AI and its explainable variants, which ensure transparency and interpretability, are increasingly being used as recognised problem-solving tools throughout the breeding process. Given these rapid technological advancements, this review provides an overview of state-of-the-art methods and processes underlying each MAS, from phenotyping, genotyping and association analyses to the integration of explainable AI along the entire workflow. In this context, we specifically address the challenges and importance of breeding winter wheat for greater drought tolerance with stable yields, as regional droughts during critical developmental stages pose a threat to winter wheat production. Finally, we explore the transition from scientific progress to practical implementation and discuss ways to bridge the gap between cutting-edge developments and breeders, expediting MAS-based winter wheat breeding for drought tolerance.

show abstract

“…CBMs like Concept-Bottleneck Models [18], Concept Whitening [27], and GlanceNets [19], among others [47,92,93] define a loss training penalty, for instance a cross-entropy loss, encouraging the extracted concepts to predict the annotations. Recently, these methods have been extended also to graph neural networks [94,95]. This solution seems straightforward: there is no more direct way than concept supervision to guide the model toward acquiring representations with the intended semantics.…”

Section: Supervised Strategiesmentioning

confidence: 99%

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning

Marconato,

Passerini,

Teso

2023

Entropy

View full text Add to dashboard Cite

Research on Explainable Artificial Intelligence has recently started exploring the idea of producing explanations that, rather than being expressed in terms of low-level features, are encoded in terms of interpretable concepts learned from data. How to reliably acquire such concepts is, however, still fundamentally unclear. An agreed-upon notion of concept interpretability is missing, with the result that concepts used by both post hoc explainers and concept-based neural networks are acquired through a variety of mutually incompatible strategies. Critically, most of these neglect the human side of the problem: a representation is understandable only insofar as it can be understood by the human at the receiving end. The key challenge in human-interpretable representation learning (hrl) is how to model and operationalize this human element. In this work, we propose a mathematical framework for acquiring interpretable representations suitable for both post hoc explainers and concept-based neural networks. Our formalization of hrl builds on recent advances in causal representation learning and explicitly models a human stakeholder as an external observer. This allows us derive a principled notion of alignment between the machine’s representation and the vocabulary of concepts understood by the human. In doing so, we link alignment and interpretability through a simple and intuitive name transfer game, and clarify the relationship between alignment and a well-known property of representations, namely disentanglement. We also show that alignment is linked to the issue of undesirable correlations among concepts, also known as concept leakage, and to content-style separation, all through a general information-theoretic reformulation of these properties. Our conceptualization aims to bridge the gap between the human and algorithmic sides of interpretability and establish a stepping stone for new research on human-interpretable representations.

show abstract

Generating Explanations for Conceptual Validation of Graph Neural Networks: An Investigation of Symbolic Predicates Learned on Relevance-Ranked Sub-Graphs

Cited by 12 publications

References 41 publications

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Robust appearance modeling for object detection and tracking: a survey of deep learning approaches

Reviewing the essential roles of remote phenotyping, GWAS and explainable AI in practical marker-assisted selection for drought-tolerant winter wheat breeding

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning

Contact Info

Product

Resources

About