I/O performance of the Santos Dumont supercomputer

Abstract

In this article, we study the I/O performance of the Santos Dumont supercomputer, since the gap between processing and data access speeds causes many applications to spend a large portion of their execution on I/O operations. For a large-scale expensive supercomputer, it is essential to ensure applications achieve the best I/O performance to promote efficient usage. We monitor a week of the machine’s activity and present a detailed study on the obtained metrics, aiming at providing an understanding of its workload. From experiences with one numerical simulation, we identified large I/O performance differences between the MPI implementations available to users. We investigated the phenomenon and narrowed it down to collective I/O operations with small request sizes. For these, we concluded that the customized MPI implementation by the machine’s vendor (used by more than 20% of the jobs) presents the worst performance. By investigating the issue, we provide information to help improve future MPI-IO collective write implementations and practical guidelines to help users and steer future system upgrades. Finally, we discuss the challenge of describing applications I/O behavior without depending on information from users. That allows for identifying the application’s I/O bottlenecks and proposing ways of improving its I/O performance. We propose a methodology to do so, and use GROMACS, the application with the largest number of jobs in 2017, as a case study.

Caracterização de Operações de E/S por meio da Análise de Logs

“…Como pode-se notar, o tamanho dos acessos varia entre 0-1 MB, sendo que a maior quantidade de acessos são operac ¸ões de leitura de até 100 bytes, o que mostra que este algoritmo de Classificac ¸ão de Imagens realiza uma série de pequenos acessos e raramente executa operac ¸ões de escrita consideráveis. Em aplicac ¸ões HPC tradicionais ocorre o inverso, são realizadas menos operac ¸ões de E/S, porém, com tamanhos maiores [Bez et al 2020].…”

I/O performance of multiscale finite element simulations on HPC environments

“…Clients use the version 2.4.3 of the Lustre client. This machine's I/O infrastructure was studied by Bez et al [20]. By default, its Lustre deployment does not stripe files (each file is fully stored in a single storage target).…”

A Arquitetura do Supercomputador SDumont e os Desafios da Pesquisa Brasileira na Área de Computação de Alto Desempenho

“…Para fornecer informações valiosas que podem ser levadas em consideração pelos administradores para promover o uso eficiente do sistema e para orientar futuras atualizações, estudamos a carga de trabalho de E/S do SDumont. Ao ajudar os usuários a obter melhor desempenho para seus aplicativos, promovemos um melhor uso da máquina [Bez et al 2020]. Existe o desafio de ajudar os usuários a usar a máquina de maneira eficiente o que depende muitas vezes do tipo de dados, experimentos e aplicações a serem acoplados na infraestrutura do SDumont.…”