Reducing the number of response time SLO violations by a cloud-HPC convergence scheduler

Kraemer, Alessandro; Maziero, Carlos; Richard, Olivier; Trystram, Denis

doi:10.1109/cloudtech.2016.7847712

Cited by 3 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Everyday usage of supercomputer centres shows the need for the separation of the cloud-like jobs (that do not require a high-bandwidth low-latency interconnect between nodes) from the multi-node parallel jobs (Caruso et al, 2005; Murli et al, 2017). Such a separation is a way for increasing efficiency of supercomputer deployment (Kraemer et al, 2016). There are attempts of statistical analysis of supercomputers operation (e.g.…”

Section: Related Workmentioning

confidence: 99%

Angara interconnect makes GPU-based Desmos supercomputer an efficient tool for molecular dynamics calculations

Stegaĭlov

Dlinnova

Ismagilov³

et al. 2019

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

In this article, we describe the Desmos supercomputer that consists of 32 hybrid nodes connected by a low-latency high-bandwidth Angara interconnect with torus topology. This supercomputer is aimed at cost-effective classical molecular dynamics calculations. Desmos serves as a test bed for the Angara interconnect that supports 3-D and 4-D torus network topologies and verifies its ability to unite massively parallel programming systems speeding-up effectively message-passing interface (MPI)-based applications. We describe the Angara interconnect presenting typical MPI benchmarks. Desmos benchmarks results for GROMACS, LAMMPS, VASP and CP2K are compared with the data for other high-performance computing (HPC) systems. Also, we consider the job scheduling statistics for several months of Desmos deployment.

show abstract

Section: Related Workmentioning

confidence: 99%

Angara interconnect makes GPU-based Desmos supercomputer an efficient tool for molecular dynamics calculations

Stegaĭlov

Dlinnova

Ismagilov³

et al. 2019

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

show abstract

“…This paper is an extended version of Kraemer et al; it brings related work on SLA‐aware scheduling strategies, a deeper description of the CArS scheduler model, more details about its validation, and new experimental results. More experiments were performed considering the same amount of low‐priority cloud jobs in Google public workload traces, and we also used a new production HPC workload trace with the same date of those workload traces.…”

Section: Introductionmentioning

confidence: 99%

Reducing the number of response time service level objective violations by a cloud‐HPC convergence scheduler

Kraemer

Maziero

Richard

et al. 2017

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Summary Job scheduling is an old topic in High‐Performance Computing (HPC), and it is more and more studied in data centers. Large data centers are often split into separate partitions for cloud computing and HPC; each partition normally has its specific scheduler. The possibility of migrating jobs from the HPC partition to the cloud one is a topic widely discussed in the literature. However, job migration from cloud to HPC is a much less explored topic. Nevertheless, such migration may be useful in many situations, in particular when the HPC platform has a low resource usage level, and the cloud usage level is high. A large number of jobs that could migrate from the cloud to the HPC partition may be observed in Google data center workloads. Job scheduling using overbooking strategy is seen as the main reason for the high resource usage level in clouds. However, overbooking can lead to a high rate of rescheduling and job dumping, which potentially causes response time violations. This work shows that HPC platforms can host and execute some cloud jobs with low interference in HPC jobs and a low number of response time violations. We introduce the definition of a cloud‐HPC convergence area and propose a job scheduling strategy for it, aiming at reducing the number of response time violations of cloud jobs without interfering with HPC jobs execution. Our proposal is formally defined and then evaluated in different execution scenarios, using the SimGrid simulation framework, with workload data from production HPC grid. The experimental results show that often, there is a large number of empty areas in the scheduling plan of HPC platforms, which makes it possible to allocate cloud jobs by backfilling. This is due to the sparse HPC job submission pattern and the low resource usage level in some HPC platforms. One performed simulation scenario considered a set of 11K parallel HPC jobs running on a 2560‐processor platform having an average resource usage level of 38.0%. The proposed convergence scheduler succeeded to inject around 267K cloud jobs in the HPC platform, with a response time violation rate under 0.00094% for such jobs, considering 80 processors in the convergence area and no effects on the HPC workload. This experiment considered cloud jobs based on job features of Google public cloud workloads, with a processing time slack factor of 1.25 (which is considered as high priority in the Google cloud SLA—Service Level Agreement). Usually, most cloud jobs show a slack factor higher than 1.25 (most cloud jobs are medium or low priority). The same simulation, repeated with a higher slack factor (4), showed no response time violations.

show abstract

Hybrid Supercomputer Desmos with Torus Angara Interconnect: Efficiency Analysis and Optimization

Kondratyuk

Smirnov

Dlinnova

et al. 2018

Communications in Computer and Information Science

View full text Add to dashboard Cite

Reducing the number of response time SLO violations by a cloud-HPC convergence scheduler

Cited by 3 publications

References 21 publications

Angara interconnect makes GPU-based Desmos supercomputer an efficient tool for molecular dynamics calculations

Angara interconnect makes GPU-based Desmos supercomputer an efficient tool for molecular dynamics calculations

Reducing the number of response time service level objective violations by a cloud‐HPC convergence scheduler

Hybrid Supercomputer Desmos with Torus Angara Interconnect: Efficiency Analysis and Optimization

Contact Info

Product

Resources

About