Binarized graph neural network

Wang, Hanchen; Lian, Defu; Zhang, Ying; Qin, Lu; He, Xiangjian; Lin, Yan; Lin, Xuemin

doi:10.1007/s11280-021-00878-3

Cited by 22 publications

(10 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[34] proposes an analysis that provides insights into better extracting and fusing information from the protein-protein interaction network for drug repurposing. Furthermore, [7,14,15,20,35] designed representation learning models [29,36] that are able to preserve both positive and negative link information within the signed graphs. These methodologies are lacking the capability to fully exploit the structural and attribute information that lies in the signed bipartite graphs.…”

Section: Gnns On Bipartite Graphmentioning

confidence: 99%

Polarity-based graph neural network for sign prediction in signed bipartite graphs

Zhang

Wang

et al. 2022

World Wide Web

Self Cite

View full text Add to dashboard Cite

As a fundamental data structure, graphs are ubiquitous in various applications. Among all types of graphs, signed bipartite graphs contain complex structures with positive and negative links as well as bipartite settings, on which conventional graph analysis algorithms are no longer applicable. Previous works mainly focus on unipartite signed graphs or unsigned bipartite graphs separately. Several models are proposed for applications on the signed bipartite graphs by utilizing the heuristic structural information. However, these methods have limited capability to fully capture the information hidden in such graphs. In this paper, we propose the first graph neural network on signed bipartite graphs, namely Polarity-based Graph Convolutional Network (PbGCN), for sign prediction task with the help of balance theory. We introduce the novel polarity attribute to signed bipartite graphs, based on which we construct one-mode projection graphs to allow the GNNs to aggregate information between the same type nodes. Extensive experiments on five datasets demonstrate the effectiveness of our proposed techniques.

show abstract

Section: Gnns On Bipartite Graphmentioning

confidence: 99%

Polarity-based graph neural network for sign prediction in signed bipartite graphs

Zhang

Wang

et al. 2022

World Wide Web

Self Cite

View full text Add to dashboard Cite

show abstract

“…Degree-Quant [317] performs quantization-aware training on graphs, which results in INT8 models often performing as well as their FP32 counterparts. BGN [318] learns binarized parameters and enables GNNs to learn discrete embedding. Bi-GCN [320] binarizes both the network parameters and the node attributes and can significantly reduce the memory consumptions by 30x for both the network parameters and node attributes, and accelerate the inference by about 47x.…”

Section: Quantized Gnnmentioning

confidence: 99%

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Yao¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Influenced by the great success of deep learning via cloud computing and the rapid development of edge chips, research in artificial intelligence (AI) has shifted to both of the computing paradigms, i.e., cloud computing and edge computing. In recent years, we have witnessed significant progress in developing more advanced AI models on cloud servers that surpass traditional deep learning models owing to model innovations (e.g., Transformers, Pretrained families), explosion of training data and soaring computing capabilities. However, edge computing, especially edge and cloud collaborative computing, are still in its infancy to announce their success due to the resource-constrained IoT scenarios with very limited algorithms deployed. In this survey, we conduct a systematic review for both cloud and edge AI. Specifically, we are the first to set up the collaborative learning mechanism for cloud and edge modeling with a thorough review of the architectures that enable such mechanism. We also discuss potentials and practical experiences of some on-going advanced edge AI topics including pretraining models, graph neural networks and reinforcement learning. Finally, we discuss the promising directions and challenges in this field.

show abstract

“…In [47], the authors propose to binarize the Graph Attention (GAT) operator [45], and evaluate their method on smallscale datasets such as Cora [37] and Pubmed [29]. In [48], the authors apply the XNOR-Net method to the GCN model [29] with success, but also on small-scale datasets.…”

Section: Model Compression In Geometric Deep Learningmentioning

confidence: 99%

“…Balance functions Recent work [42] has uncovered possible limitations in binary graph and point cloud learning models when quantizing the output of max-pooling aggregation of batch-normalized high-dimensional features. Similarly, [47] claim that a balance function is necessary to avoid large values in the outputs of the dot product operations when most pre-quantization inputs are positive. We evaluate two strategies for re-centering the input of sign after max aggregation, namely mean-centering, and median-centering (thus ensuring a perfectly balanced distribution of positive and negative values pre-quantization).…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…We benchmark our binarized GraphSAGE on the OGB-Product and OGB-Protein node property prediction datasets [23], which are recent and challenging (2,449,029 nodes, 61,859,140 edges for OGB-product) benchmarks with standardized evaluation procedures, compared to the more commonly used ones, such as Cora [37] (2708 nodes, 5429 edges) used in [47] or Reddit [21] (232,965 nodes, 114,615,892 edges) used in [48]. Notably, the Protein dataset is challenging due to the high average node degree and small graph diameter which may exacerbates limitations of GNNs [1].…”

Section: Large-scale Inductive Learning With Graphsagementioning

confidence: 99%

See 1 more Smart Citation

Binary Graph Neural Networks

Bahri¹,

Bahl²,

Zafeiriou³

2020

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have emerged as a powerful and flexible framework for representation learning on irregular data. As they generalize the operations of classical CNNs on grids to arbitrary topologies, GNNs also bring much of the implementation challenges of their Euclidean counterparts. Model size, memory footprint, and energy consumption are common concerns for many realworld applications. Network binarization allocates a single bit to network parameters and activations, thus dramatically reducing the memory requirements (up to 32x compared to single-precision floating-point parameters) and maximizing the benefits of fast SIMD instructions of modern hardware for measurable speedups. However, in spite of the large body of work on binarization for classical CNNs, this area remains largely unexplored in geometric deep learning. In this paper, we present and evaluate different strategies for the binarization of graph neural networks. We show that through careful design of the models, and control of the training process, binary graph neural networks can be trained at only a moderate cost in accuracy on challenging benchmarks. In particular, we present the first dynamic graph neural network in Hamming space, able to leverage efficient k-NN search on binary vectors to speed-up the construction of the dynamic graph. We further verify that the binary models offer significant savings on embedded devices.

show abstract

Binarized graph neural network

Cited by 22 publications

References 28 publications

Polarity-based graph neural network for sign prediction in signed bipartite graphs

Polarity-based graph neural network for sign prediction in signed bipartite graphs

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Binary Graph Neural Networks

Contact Info

Product

Resources

About