Interpretable Deep Graph Generation with Node-edge Co-disentanglement

Abstract

Disentangled representation learning has recently attracted significant amount of attentions, particularly in the field of image representation learning. However, learning the disentangled representations behind a graph remains largely unexplored, especially for the attributed graph with both node and edge features. Disentanglement learning for graph generation has substantial new challenges including: 1) the lack of graph deconvolution operations to jointly decode node and edge attributes; and 2) the difficulty in enforcing the disentanglement among latent factors that respectively influence: i) only nodes, ii) only edges, and iii) joint patterns between them. To address these challenges, we propose a new disentanglement enhancement framework for deep generative models for attributed graphs. In particular, a novel variational objective is proposed to disentangle the above three types of latent factors, with novel architecture for node and edge deconvolutions. Moreover, within each type, individual-factor-wise disentanglement is further enhanced, which is shown to be a generalization of existing framework for images. Qualitative and quantitative experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed model and its extensions. CCS CONCEPTS• Computing methodologies → Unsupervised learning; Neural networks; Generative and developmental approaches; • Mathematics of computing → Graph algorithms; • Information systems → Data mining; • Networks → Topology analysis and generation; • Applied computing → Molecular structural biology.

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

“…Examples of such interpretable factors include color of an object in an image, or the presence of a smile on a face [66]. This can be quantified by measuring the mutual information between a dimension and a concept [89]. Ideally, a dimension has high mutual information with a single concept and zero mutual information with all other concepts.…”

“…Others vary exactly one dimension and keep all others fixed. Then, the output variance per concept can be measured [66], or the accuracy of a classifier that should predict the index of the specific factor of variation (ceteris paribus) [89]. Another approach, originally suggested by [129], is to generate data while keeping exactly one covariate fixed and varying the others, and evaluate whether the variance in one dimension is exactly zero [25,156].…”

Deep Generative Model for Periodic Graphs

“…The goal of disentangled representation learning is to separate underlying semantic factors accounting for the variation of the data in the learned representation. This disentangled representation has been shown to be resilient to the complex factors involved (Bengio et al, 2013), and can enhance the generalizability and improve robustness against adversarial attack (Guo et al, 2020a;Alemi et al, 2016). Intuitively, disentangled representation learning can achieve superior interpretability over regular graph representation learning tasks to better understand the graphs in various domains (Guo et al, 2020b).…”

“…Intuitively, disentangled representation learning can achieve superior interpretability over regular graph representation learning tasks to better understand the graphs in various domains (Guo et al, 2020b). This motivates the surge of a few VAEbased approaches that modify the VAE objective by adding, removing or adjusting the weight of individual terms for deep graph generation tasks (Guo et al, 2020a;Chen et al, 2018;Kim & Mnih, 2018). Disentangled representation learning is important in modeling periodical graphs where the process requires to distinguish the repeatable patterns from the others.…”

Recovering Barabási-Albert Parameters of Graphs through Disentanglement

“…Disentangled representations have been successfully found for image data (Burgess et al, 2019;van Steenkiste et al, 2019;Leeb et al, 2020;Besserve et al, 2019). However, disentanglement of graph data is largely unexplored, with a few exceptions (Ma et al, 2019;Guo et al, 2020;Stoehr et al, 2019).…”