Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable?

Abstract

The recent series of innovations in deep learning (DL) have shown enormous potential to impact individuals and society, both positively and negatively. The DL models utilizing massive computing power and enormous datasets have significantly outperformed prior historical benchmarks on increasingly difficult, well-defined research tasks across technology domains such as computer vision, natural language processing, signal processing, and human-computer interactions. However, the Black-Box nature of DL models and their over-reliance on massive amounts of data condensed into labels and dense representations poses challenges for interpretability and explainability of the system. Furthermore, DLs have not yet been proven in their ability to effectively utilize relevant domain knowledge and experience critical to human understanding. This aspect is missing in early data-focused approaches and necessitated knowledge-infused learning and other strategies to incorporate computational knowledge. This article demonstrates how knowledge, provided as a knowledge graph, is incorporated into DL methods using knowledge-infused learning, which is one of the strategies. We then discuss how this makes a fundamental difference in the interpretability and explainability of current approaches, and illustrate it with examples from natural language processing for healthcare and education applications.

"Is depression related to cannabis?": A knowledge-infused model for Entity and Relation Extraction with Limited Supervision

“…Wang et al [45] augment the BERT model with a structured prediction layer to predict multiple relations in one pass. In all the approaches discussed so far, knowledge has not been a component of the architecture [46].…”

Semantic XAI for contextualized demand forecasting explanations

“…Doctor XAI (Panigutti et al (2020)) develops an agnostic XAI technique for ontology-linked data classification by training a surrogate model and extracting rules from it. Gaur et al (2020) make use of a Knowledge Graph to feed deep learning models with it to enhance their explainability. Samek and Müller (2019) envision Knowledge Graphs can be used to compact large tree models by combining nodes into unique probabilistic concepts.…”

“…Such explanations can be provided on a global level (the forecasting model in general) or at a local level (for every prediction instance). Some authors also envisioned that the inclusion of semantic technologies could be used to determine the semantic closeness of concepts encoded in data features or integrate knowledge graph embeddings to a forecasting model to produce explanations along with the forecasts (Gaur et al (2020); Panigutti et al (2020)).…”