HPCGCN: A Predictive Framework on High Performance Computing Cluster Log Data Using Graph Convolutional Networks

Bose, Avishek Joey; Yang, Huichen; Hsu, William H.; Andresen, Daniel

doi:10.1109/bigdata52589.2021.9671370

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, Artificial Intelligence (AI) can provide significant benefits, as it allows to exploit the available big data effectively and to create decision support tools for HPC system administrators and facility managers [12,13]. In the past, many works from the literature and the practice demonstrated the possibility to extract useful information using data collected from HPC computing nodes and employing supervised Deep Learning (DL) models [14][15][16] and semi-supervised ones [17][18][19]. These methods have been applied to detect nodes' availability, defined as operation without anomalies.…”

Section: Introductionmentioning

confidence: 99%

Analysing Supercomputer Nodes Behaviour with the Latent Representation of Deep Learning Models

Molan

Borghesi

Benini

et al. 2022

Euro-Par 2022: Parallel Processing

View full text Add to dashboard Cite

Anomaly detection systems are vital in ensuring the availability of modern High-Performance Computing (HPC) systems, where many components can fail or behave wrongly. Building a data-driven representation of the computing nodes can help with predictive maintenance and facility management. Luckily, most of the current supercomputers are endowed with monitoring frameworks that can build such representations in conjunction with Deep Learning (DL) models. In this work, we propose a novel semi-supervised DL approach based on autoencoder networks and clustering algorithms (applied to the latent representation) to build a digital twin of the computing nodes of the system. The DL model projects the node features into a lower-dimensional space. Then, clustering is applied to capture and reveal underlying, non-trivial correlations between the features. The extracted information provides valuable insights for system administrators and managers, such as anomaly detection and node classification based on their behaviour and operative conditions. We validated the approach on 240 nodes from the Marconi 100 system, a Tier-0 supercomputer located in CINECA (Italy), considering a 10-month period.

show abstract