Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

Neuwirth, Sarah; Paul, Arnab

doi:10.1109/cluster48925.2021.00100

“…Given the complexity of modern supercomputers and HPC systems, achieving theoretical peak performance depends on myriad parameters. To optimize system and make efficient use of underlying resources, various methods can be applied [44], including simulation, benchmarking, and monitoring. However, these methods and the tools are not compatible with each other, i.e., both the individual tools and the approaches consider only a certain part of a certain domain, e.g., network, I/O, or resource allocation.…”

Section: Pos(lattice2022)245mentioning

confidence: 99%

Modular Supercomputing and its Role in Europe’s Exascale Computing Strategy

Neuwirth¹

2023

Proceedings of the 39th International Symposium on Lattice Field Theory — PoS(LATTICE2022)

3

0

View full text Add to dashboard Cite

Reaching Exascale compute performance at an affordable budget requires increasingly heterogeneous HPC systems, which combine general purpose processing units (CPUs) with acceleration devices such as graphics processing units (GPUs) or many-core processors. The Modular Supercomputing Architecture (MSA) developed within the EU-funded DEEP project series breaks with traditional HPC system architectures by orchestrating these heterogeneous computing resources at system-level, organizing them in compute modules with different hardware and performance characteristics. Modules with disruptive technologies, such as quantum devices, can also be included in a modular supercomputer to satisfy the needs of specific user communities. The goal is to provide cost-effective computing at extreme performance scales fitting the needs of a wide range of Computational Sciences. This approach brings substantial benefits for heterogeneous applications and workflows since each part can be run on exactly matching computing resources, therefore improving the time to solution and energy use. It is therefore ideal for supercomputer centers running a heterogeneous mix of applications. This work introduces the Modular Supercomputing Architecture -which constitutes the central element in Europe's roadmap to Exascale computing -, including its history, its role in Europe's Exascale computing strategy, its hardware and software elements, and experiences from mapping applications and workflows to MSA systems.

show abstract

“…Consequently, performance degradation is unavoidable at each stage. Furthermore, such a complex software stack makes debugging I/O problems and identifying the cause of performance degradation inherently challenging [19,20].…”

Section: Introductionmentioning

confidence: 99%

I/O-signature-based feature analysis and classification of high-performance computing applications

Park,

Huang,

Lee

et al. 2023

Preprint

0

View full text Add to dashboard Cite

The demand for high-performance computing (HPC) resources in computing fields such as machine learning has increased significantly in recent years. Computing power has been growing exponentially to keep up with this demand. However, these gains have not been able to translate to performance improvement in real-world applications. One of the biggest reasons for this is performance degradation in terms of input/output (I/O) due to the increased storage latency and complex parallel I/O architecture of accessing data in distributed storage systems. In this study, we analyze application-specific I/O patterns to gain a deeper understanding of I/O throughput and the interaction between applications and the I/O system. Specifically, we analyze the importance of each feature of I/O patterns through feature analysis based on the collected monitoring information. We also investigate the feasibility of identifying the application based on the analyzed key features. To this end, we present the analysis accuracy and confusion matrix of four algorithms, including random forest, which are widely used as classification algorithms in the experimental results. The experiment results confirm that we can distinguish applications with an accuracy of more than 90\% by using application-specific I/O patterns.

show abstract

I/O-signature-based feature analysis and classification of high-performance computing applications

Park,

Huang,

Lee

et al. 2023

Cluster Comput

0

View full text Add to dashboard Cite

The demand for high-performance computing (HPC) resources in computing fields such as machine learning has increased significantly in recent years. Computing power has been growing exponentially to keep up with this demand. However, these gains have not been able to translate to performance improvement in real-world applications. One of the biggest reasons for this is performance degradation in terms of input/output (I/O) due to the increased storage latency and complex parallel I/O architecture of accessing data in distributed storage systems. In this study, we analyze application-specific I/O patterns to gain a deeper understanding of I/O throughput and the interaction between applications and the I/O system. Specifically, we analyze the importance of each feature of I/O patterns through feature analysis based on the collected monitoring information. We also investigate the feasibility of identifying the application based on the analyzed key features. To this end, we present the analysis accuracy and confusion matrix of four algorithms, including random forest, which are widely used as classification algorithms in the experimental results. The experiment results confirm that we can distinguish applications with an accuracy of more than 90% by using application-specific I/O patterns.

show abstract

Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

Cited by 13 publications

References 58 publications

Modular Supercomputing and its Role in Europe’s Exascale Computing Strategy

Modular Supercomputing and its Role in Europe’s Exascale Computing Strategy

I/O-signature-based feature analysis and classification of high-performance computing applications

I/O-signature-based feature analysis and classification of high-performance computing applications

Contact Info

Product

Resources

About