Influence of Tasks Duration Variability on Task-Based Runtime Schedulers

Abstract

In the context of HPC platforms, individual nodes nowadays consist in heterogenous processing resources such as GPU units and multicores. Those resources share communication and storage resources, inducing complex co-scheduling effects, and making it hard to predict the exact duration of a task or of a communication. To cope with these issues, runtime dynamic schedulers such as StarPU have been developed. These systems base their decisions at runtime on the state of the platform and possibly on static priorities of tasks computed offline. In this paper, our goal is to quantify performance variability in the context of HPC heterogeneous nodes, by focusing on very regular dense linear algebra kernels. Then, we analyze the impact of this variability on a dynamic runtime scheduler such as StarPU, in order to analyze whether the strategies that have been designed in the context of MapReduce applications to cope with stragglers could be transferred to HPC systems, or if the dynamic nature of runtime schedulers is enough to cope with actual performance variations.

READYS: A Reinforcement Learning Based Strategy for Heterogeneous Dynamic Scheduling

“…This is typically the case in practice, due to the suitability of a given kernel on one particular computing unit, or another. Nevertheless, in practice, due to heating conditions or NUMA effects, the duration of the execution of a given task on a given resource is not constant and the variability in the processing time also depends on the resource on which they are performed [11].…”

An approach for realistically simulating the performance of scientific applications on high performance computing systems

“…The estimated P L is calculated as in Equation 3 and is used to disturb the processor availability during simulation, i.e. :…”

“…Realistically simulating application performance is, however, nontrivial. Several studies addressed the topic of application performance simulation for specific purposes, such as evaluating the performance of scheduling techniques under variable task execution times with a specific runtime system [3], or focusing on improving communications in large and distributed applications [4].…”

Detecção de Anomalias de Desempenho em Aplicações de Alto Desempenho baseadas em Tarefas em Clusters Híbridos

“…No contexto de aplicações com tarefas em arquiteturas híbridas, nós apresentamos previamente uma discussão preliminar dos experimentos aqui apresentados [Schnorr et al 2017]. Uma análise similar, utilizando a mesma aplicação (cholesky do pacote Chameleon) e o mesmo ambiente de execução (StarPU) foi feita por [Beaumont et al 2018]. Neste estudo, o objetivo dos autores foi avaliar o impacto da presença de tarefas com duração atípica no tempo total de execução da aplicação (makespan).…”