Improving Graph Neural Networks with Structural Adaptive Receptive Fields

Ma, Xiaojun; Wang, Junshan; Chen, Han Y. H.; Song, Guojie

doi:10.1145/3442381.3449896

Cited by 14 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these GNNs focus on fixed graphs with feature-level similarities and cannot adapt to spatiotemporal graphs. To empower an adaptive topology, AGCRN [31] learns a dynamic topology based on feature-wise product, while STAR-GNN [46] and L2P framework [47] respectively design a mutual-information based strategy and a generative process, to estimate the optimal receptive fields for GNNs. More recently, exogenous context factors those are out-of-graph have been demonstrated to potentially interfere the intrinsic graph topology and have complicated interactions on element-wise aggregations [25], [27], [48].…”

Section: F Hyperparameter Settingsmentioning

confidence: 99%

Towards Learning in Grey Spatiotemporal Systems: A Prophet to Non-consecutive Spatiotemporal Dynamics

Zhou¹,

Kuo²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Spatiotemporal forecasting is an imperative topic in data science due to its diverse and critical applications in smart cities. Existing works mostly perform consecutive predictions of following steps with observations completely and continuously obtained, where nearest observations can be exploited as key knowledge for instantaneous status estimation. However, the practical issues of early activity planning and sensor failures elicit a brand-new task, i.e., non-consecutive forecasting. In this paper, we define spatiotemporal learning systems with missing observation as Grey Spatiotemporal Systems (G2S) and propose a Factor-Decoupled learning framework for G2S (FDG2S), where the core idea is to hierarchically decouple multi-level factors and enable both flexible aggregations and disentangled uncertainty estimations. Firstly, to compensate for missing observations, a generic semantic-neighboring sequence sampling is devised, which selects representative sequences to capture both periodical regularity and instantaneous variations. Secondly, we turn the predictions of non-consecutive statuses into inferring statuses under expected combined exogenous factors. In particular, a factor-decoupled aggregation scheme is proposed to decouple factor-induced predictive intensity and region-wise proximity by two energy functions of conditional random field. To infer region-wise proximity under flexible factor-wise combinations and enable dynamic neighborhood aggregations, we further disentangle compounded influences of exogenous factors on regionwise proximity and learn to aggregate them. Given the inherent incompleteness and critical applications of G2S, a DisEntangled Uncertainty Quantification is put forward, to identify two types of uncertainty for reliability guarantees and model interpretations.Extensive experiments demonstrate that our solution can promote the performance by at least 8.50% on early planning and 2.01%-18.00% on the sensor failure setting.

show abstract

Section: F Hyperparameter Settingsmentioning

confidence: 99%

Towards Learning in Grey Spatiotemporal Systems: A Prophet to Non-consecutive Spatiotemporal Dynamics

Zhou¹,

Kuo²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…AGFormer (Jiang et al 2023) first selects representative anchors and then transforms node-to-node message passing into an anchor-to-anchor and anchor-to-node message passing process. Moreover, AGT (Ma et al 2023b) extracts the structural patterns from subgraph views and designs an adaptive transformer block to dynamically integrate attention scores in a node-specific manner.…”

Section: Introductionmentioning

confidence: 99%

Dual-channel graph contrastive learning for self-supervised graph-level representation learning

Luo

Dong

Zheng

et al. 2023

Pattern Recognition

View full text Add to dashboard Cite

“…Graph neural networks (GNNs) [56] have become the state-of-theart methods in many graph representation learning scenarios such as node classification [8,31,32,61], link prediction [3,14,47,54], recommendation [17,34,53,58], and knowledge graphs [1,48,49,51]. Most GNN pipelines can be described in terms of the neural message passing (NMP) framework [15], which is based on the core idea of recursive neighborhood aggregation and transformation.…”

Section: Introductionmentioning

confidence: 99%

PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm

Zhang,

Shen,

Lin

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have achieved state-of-the-art performance in various graph-based tasks. However, as mainstream GNNs are designed based on the neural message passing mechanism, they do not scale well to data size and message passing steps. Although there has been an emerging interest in the design of scalable GNNs, current researches focus on specific GNN design, rather than the general design space, limiting the discovery of potential scalable GNN models. This paper proposes PaSca, a new paradigm and system that offers a principled approach to systemically construct and explore the design space for scalable GNNs, rather than studying individual designs. Through deconstructing the message passing mechanism, PaSca presents a novel Scalable Graph Neural Architecture Paradigm (SGAP), together with a general architecture design space consisting of 150k different designs. Following the paradigm, we implement an auto-search engine that can automatically search well-performing and scalable GNN architectures to balance the trade-off between multiple criteria (e.g., accuracy and efficiency) via multi-objective optimization. Empirical studies on ten benchmark datasets demonstrate that the representative instances (i.e., PaSca-V1, V2, and V3) discovered by our system achieve consistent performance among competitive baselines. Concretely, PaSca-V3 outperforms the state-of-the-art GNN method JK-Net by 0.4% in terms of predictive accuracy on our large industry dataset while achieving up to 28.3× training speedups. CCS CONCEPTS• Mathematics of computing → Graph algorithms.

show abstract

Improving Graph Neural Networks with Structural Adaptive Receptive Fields

Cited by 14 publications

References 12 publications

Towards Learning in Grey Spatiotemporal Systems: A Prophet to Non-consecutive Spatiotemporal Dynamics

Towards Learning in Grey Spatiotemporal Systems: A Prophet to Non-consecutive Spatiotemporal Dynamics

Dual-channel graph contrastive learning for self-supervised graph-level representation learning

PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm

Contact Info

Product

Resources

About