ML-Based Methodology for HPC Facilities Supervision

Anton, Laetitia; Willemot, Sophie; Gougeaud, Sebastien; Zertal, Soraya

doi:10.1007/978-3-031-40843-4_23

Cited by 1 publication

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using Grafana, we created custom dashboards to visualize the collected systemlevel metrics in real-time. The dashboards provided insights into CPU utilization trends, memory usage patterns, disk activity, and network throughput across different nodes within the cluster [11]. This visualization enabled us to identify performance bottlenecks, optimize resource allocation, and enhance overall system efficiency.…”

Section: Agent-based System Monitoring With Prometheus Node Exporter ...mentioning

confidence: 99%

Real-time Performance Monitoring of HPC Clusters: Techniques and Challenges

Kamath

2024

IJSREM

View full text Add to dashboard Cite

In the era of exascale computing, High-Performance Computing (HPC) clusters have become essential for addressing complex computational challenges across various domains. The increasing scale and complexity of HPC systems pose significant challenges in ensuring optimal performance and resource utilization. Real-time performance monitoring has emerged as a critical requirement to address these challenges effectively. The paper offers a comprehensive analysis of real-time performance monitoring techniques and challenges within HPC clusters focusing on three key monitoring approaches: Prometheus for agent-based monitoring, the ELK (Elasticsearch, Logstash, Kibana) stack for log-based monitoring, and machine learning/artificial intelligence techniques for proactive analysis. The implementation of Prometheus and Node Exporter for real-time metrics collection, the utilization of the ELK stack for log aggregation and analysis, and the integration of ML/AI for anomaly detection and predictive analytics are explored. Challenges such as scalability, heterogeneity, and dynamic workloads are addressed in the context of each monitoring approach. The findings demonstrate that each technique offers unique advantages and trade-offs, providing valuable insights for improving HPC performance monitoring practices. Key Words: High-Performance Computing, real-time performance monitoring, Prometheus, ELK stack, Machine Learning, Artificial Intelligence, HPC clusters.

show abstract

Section: Agent-based System Monitoring With Prometheus Node Exporter ...mentioning

confidence: 99%

Real-time Performance Monitoring of HPC Clusters: Techniques and Challenges

Kamath

2024

IJSREM

View full text Add to dashboard Cite

show abstract

ML-Based Methodology for HPC Facilities Supervision

Cited by 1 publication

References 19 publications

Real-time Performance Monitoring of HPC Clusters: Techniques and Challenges

Real-time Performance Monitoring of HPC Clusters: Techniques and Challenges

Contact Info

Product

Resources

About