Inclusion of domain-knowledge into GNNs using mode-directed inverse entailment

Abstract

We present a general technique for constructing Graph Neural Networks (GNNs) capable of using multi-relational domain knowledge. The technique is based on mode-directed inverse entailment (MDIE) developed in Inductive Logic Programming (ILP). Given a data instance e and background knowledge B, MDIE identifies a most-specific logical formula ⊥ B (e) that contains all the relational information in B that is related to e. We represent ⊥ B (e) by a "bottom-graph" that can be converted into a form suitable for GNN implementations. This transformation allows a principled way of incorporating generic background knowledge into GNNs: we use the term 'BotGNN' for this form of graph neural networks. For several GNN variants, using real-world datasets with substantial background knowledge, we show that BotGNNs perform significantly better than both GNNs without background knowledge and a recently proposed simplified technique for including domain knowledge into GNNs. We also provide experimental evidence comparing BotGNNs favourably to multi-layer perceptrons that use features representing a "propositionalised" form of the background knowledge; and BotGNNs to a standard ILP based on the use of most-specific clauses. Taken together, these results point to BotGNNs as capable of combining the computational efficacy of GNNs with the representational versatility of ILP.

Composition of Relational Features with an Application to Explaining Black-Box Predictors

“…This suggests that substantially more experimentation is needed to see if the predictive performance of Simple CRMs can be improved. We note also that that the DRM uses substantially more complex features than the Simple CRM, and that (Dash et al, 2022); DRM (500), is a form of MLP called a Deep Relational Machine that has as input Boolean feature-vectors resulting from a stochastic selection of 500 relational features (Dash et al, 2019); and CILP++, an MLP that has input Boolean feature-vectors resulting from an exhaustive feature construction technique called Bottom-Clause Propositionalisation. Baseline is the majority class predictor.…”

“…There are also higher-level domain-relations that determine presence of connected, fused structures. Some more details on the background-knowledge can be seen in these recent studies: (Dash et al, 2021(Dash et al, , 2022. 2021) is taken as the target model.…”

“…For the real-world datasets, the current state-of-the-art predictions are from a Graph Neural Network (GNN) constructed using the background knowledge described earlier (Dash et al, 2022). However, the GNN model constitutes a black-box model, since it does not produce any explanations for its predictions.…”

“…We borrow some of the following definitions from (Dash et al, 2022). The definition of λµ sequence is simplified as we are dealing only with feature clauses here and all other definitions are same as in (Dash et al, 2022).…”

One-way Explainability Isn't The Message

“…We show first recent results reported in [10,11]. The experiments reported consider the inclusion of human-selected domain-knowledge for two kinds of deep neural networks (a multi-layer perceptron, or MLP, and a graph neural network, or GNN).…”