Profiling Platform Storage Using IO500 and Mistral

Monnier, Nolan; Lofstead, Jay; Lawson, Margaret; Curry, Matthew

doi:10.1109/pdsw49588.2019.00011

Cited by 4 publications

(2 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After Linpack, the most popular benchmarks in the HPC community target specific architecture features: HPCG [11], NAS parallel benchmarks [4] or IO500 [24]. For the purpose of evaluating new architectures, these benchmarks are too specific [22], which calls for a bottom up approach.…”

Section: Related Workmentioning

confidence: 99%

Size & Shape Matters: The Need of HPC Benchmarks of High Resolution Image Training for Deep Learning

Pont

Megias

Garcia-Gasulla

et al. 2021

JSFI

View full text Add to dashboard Cite

One of the purposes of HPC benchmarks is to identify limitations and bottlenecks in hardware. This functionality is particularly influential when assessing performance on emerging tasks, the nature and requirements of which may not yet be fully understood. In this setting, a proper benchmark can steer the design of next generation hardware by properly identifying said requirements, and quicken the deployment of novel solutions. With the increasing popularity of deep learning workloads, benchmarks for this family of tasks have been gaining popularity. Particularly for image based tasks, which rely on the most well established family of deep learning models: Convolutional Neural Networks. Significantly, most benchmarks for CNN use low-resolution and fixed-shape (LR&FS) images. While this sort of inputs have been very successful for certain purposes, they are insufficient for some domains of special interest (e.g., medical image diagnosis or autonomous driving) where one requires higher resolutions and variable-shape (HR&VS) images to avoid loss of information and deformation. As of today, it is still unclear how does image resolution and shape variability affect the nature of the problem from a computational perspective. In this paper we assess the differences between training with LR&FS and HR&VS, as means to justify the importance of building benchmarks specific for the latter. Our results on three different HPC clusters show significant variations in time, resources and memory management, highlighting the differences between LR&FS and HR&VS image deep learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Size & Shape Matters: The Need of HPC Benchmarks of High Resolution Image Training for Deep Learning

Pont

Megias

Garcia-Gasulla

et al. 2021

JSFI

View full text Add to dashboard Cite

show abstract

“…Two lists of IO500 results are published: The "Full List" or "Ranked List" contains performance results that are achieved on an arbitrary number of client nodes. The "10 Node Challenge" list contains results for exactly 10 client nodes, which provides a standardized basis for comparing those IO500 workloads which scale with the number of client nodes [3]. For both lists, there are no constraints regarding the size of the storage system.…”

Section: Performancementioning

confidence: 99%

DAOS: A Scale-Out High Performance Storage Stack for Storage Class Memory

Liang¹,

Lombardi²,

Chaarawi

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The Distributed Asynchronous Object Storage (DAOS) is an open source scale-out storage system that is designed from the ground up to support Storage Class Memory (SCM) and NVMe storage in user space. Its advanced storage API enables the native support of structured, semi-structured and unstructured data models, overcoming the limitations of traditional POSIX based parallel filesystem. For HPC workloads, DAOS provides direct MPI-IO and HDF5 support as well as POSIX access for legacy applications. In this paper we present the architecture of the DAOS storage engine and its high-level application interfaces. We also describe initial performance results of DAOS for IO500 benchmarks.

show abstract