Characterizing and Understanding Distributed GNN Training on GPUs

Lin, Haiyang; Yan, Mingyu; Yang, Xiaocheng; Zou, Mo; Li, Wenming; Ye, Xiaochun; Fan, Dongrui

doi:10.1109/lca.2022.3168067

Cited by 6 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [39] provided a detailed workload analysis on the inference of GNNs. Lin et al [27] focused on the distributed training benchmark of three GNN models implemented in PyG. The studies in [3,22,30] presented new datasets for GNN benchmarking.…”

Section: Related Workmentioning

confidence: 99%

Characterizing the Efficiency of Graph Neural Network Frameworks with a Magnifying Glass

Huang¹,

Kim²,

Rees³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have received great attention due to their success in various graph-related learning tasks. Several GNN frameworks have then been developed for fast and easy implementation of GNN models. Despite their popularity, they are not well documented, and their implementations and system performance have not been well understood. In particular, unlike the traditional GNNs that are trained based on the entire graph in a full-batch manner, recent GNNs have been developed with di erent graph sampling techniques for mini-batch training of GNNs on large graphs. While they improve the scalability, their training times still depend on the implementations in the frameworks as sampling and its associated operations can introduce nonnegligible overhead and computational cost. In addition, it is unknown how much the frameworks are 'eco-friendly' from a green computing perspective. In this paper, we provide an indepth study of two mainstream GNN frameworks along with three state-of-the-art GNNs to analyze their performance in terms of runtime and power/energy consumption. We conduct extensive benchmark experiments at several di erent levels and present detailed analysis results and observations, which could be helpful for further improvement and optimization.

show abstract