HPC AI500: A Benchmark Suite for HPC AI Systems

Jiang, Zihan; Gao, Wanling; Wang, Lei; Xiong, Xingwang; Zhang, Yuchen; Xu, Wen; Luo, Chunjie; Ye, Hainan; Li, Xiaoxue; Zhang, Yunquan; Shi, Feng; Li, Kenli; Xu, Wei; Zhan, Jianfeng

doi:10.1007/978-3-030-32813-9_2

Cited by 30 publications

(12 citation statements)

References 42 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context and with the growing popularity of AI applicaions, several HPC benchmarks for DNNs have been released. This includes AI500, [15], where high resolution images are used on weather predictions; HPL-AI, where mixed-precision operations are tested through solving a system of linear equations, [17,18] or MLBench, on which different datasets from diverse fields can be processed, [20].…”

Section: Related Workmentioning

confidence: 99%

Size & Shape Matters: The Need of HPC Benchmarks of High Resolution Image Training for Deep Learning

Pont

Megias

Garcia-Gasulla

et al. 2021

JSFI

View full text Add to dashboard Cite

One of the purposes of HPC benchmarks is to identify limitations and bottlenecks in hardware. This functionality is particularly influential when assessing performance on emerging tasks, the nature and requirements of which may not yet be fully understood. In this setting, a proper benchmark can steer the design of next generation hardware by properly identifying said requirements, and quicken the deployment of novel solutions. With the increasing popularity of deep learning workloads, benchmarks for this family of tasks have been gaining popularity. Particularly for image based tasks, which rely on the most well established family of deep learning models: Convolutional Neural Networks. Significantly, most benchmarks for CNN use low-resolution and fixed-shape (LR&FS) images. While this sort of inputs have been very successful for certain purposes, they are insufficient for some domains of special interest (e.g., medical image diagnosis or autonomous driving) where one requires higher resolutions and variable-shape (HR&VS) images to avoid loss of information and deformation. As of today, it is still unclear how does image resolution and shape variability affect the nature of the problem from a computational perspective. In this paper we assess the differences between training with LR&FS and HR&VS, as means to justify the importance of building benchmarks specific for the latter. Our results on three different HPC clusters show significant variations in time, resources and memory management, highlighting the differences between LR&FS and HR&VS image deep learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Size & Shape Matters: The Need of HPC Benchmarks of High Resolution Image Training for Deep Learning

Pont

Megias

Garcia-Gasulla

et al. 2021

JSFI

View full text Add to dashboard Cite

show abstract

“…Because of the difficulty of recruiting qualified scientists to label scientific data, AI for science applications lag but is promising. In general, the scientific data are often more complicated than that of the MINST or ImageNet data: the shape of scientific data can be 2D images or higher-dimension structures with hundreds of channels, while the popular commercial image data like ImageNet often consist of only RGB [17]. So we should include the scientific data in the HPC AI benchmarks.…”

Section: The Requirements In Hpc Fieldmentioning

confidence: 99%

“…This paper presents HPC AI500-a comprehensive HPC AI benchmarking methodology, tools, and metrics. Compared to our previous position paper [17], this paper proposes a brand-new benchmarking methodology that simultaneously achieves representative, repeatable and simple.…”

Section: Introductionmentioning

confidence: 99%

HPC AI500: Representative, Repeatable and Simple HPC AI Benchmarking

Jiang,

Gao,

Tang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent years witness a trend of applying large-scale distributed deep learning algorithms (HPC AI) in both business and scientific computing areas, whose goal is to speed up the training time to achieve a state-of-the-art quality. The HPC AI benchmarks accelerate the process. Unfortunately, benchmarking HPC AI systems at scale raises serious challenges. This paper presents a representative, repeatable and simple HPC AI benchmarking methodology. Among the seventeen AI workloads of AIBench Training-by far the most comprehensive AI Training benchmarks suite-we choose two representative and repeatable AI workloads. The selected HPC AI benchmarks include both business and scientific computing: Image Classification and Extreme Weather Analytics. To rank HPC AI systems, we present a new metric named Valid FLOPS, emphasizing both throughput performance and a target quality. The specification, source code, datasets, and HPC AI500 ranking numbers are publicly available from https: //www.benchcouncil.org/HPCAI500/.

show abstract

“…Most of the aspects that the intelligence deals with rational actions are adopted. Reference [47], did not consider the use of AI in HPC to make energy efficient. They suggested HPC AI500-a test suite to analyze HPC systems that run scientific workloads on the DL.…”

Section: Comparative Studiesmentioning

confidence: 99%

Artificial Intelligence: An Energy Efficiency Tool for Enhanced High performance computing

et al. 2020

View full text Add to dashboard Cite

Power-consuming entities such as high performance computing (HPC) sites and large data centers are growing with the advance in information technology. In business, HPC is used to enhance the product delivery time, reduce the production cost, and decrease the time it takes to develop a new product. Today’s high level of computing power from supercomputers comes at the expense of consuming large amounts of electric power. It is necessary to consider reducing the energy required by the computing systems and the resources needed to operate these computing systems to minimize the energy utilized by HPC entities. The database could improve system energy efficiency by sampling all the components’ power consumption at regular intervals and the information contained in a database. The information stored in the database will serve as input data for energy-efficiency optimization. More so, device workload information and different usage metrics are stored in the database. There has been strong momentum in the area of artificial intelligence (AI) as a tool for optimizing and processing automation by leveraging on already existing information. This paper discusses ideas for improving energy efficiency for HPC using AI.

show abstract

HPC AI500: A Benchmark Suite for HPC AI Systems

Cited by 30 publications

References 42 publications

Size & Shape Matters: The Need of HPC Benchmarks of High Resolution Image Training for Deep Learning

Size & Shape Matters: The Need of HPC Benchmarks of High Resolution Image Training for Deep Learning

HPC AI500: Representative, Repeatable and Simple HPC AI Benchmarking

Artificial Intelligence: An Energy Efficiency Tool for Enhanced High performance computing

Contact Info

Product

Resources

About