Enhancing an Open Source Resource Manager with Multi-core/Multi-threaded Support

Balle, Susanne M.; Palermo, Daniel J.

doi:10.1007/978-3-540-78699-3_3

Cited by 13 publications

(7 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The select/cons res plugin is ideal for multicore and manycore architectures where nodes are viewed as a collection of consumable resources (such as cores and memory). In this plugin, nodes can be used exclusively or in a shared mode where a job may allocate its own resources differently than the other jobs sharing the same node [1].…”

Section: Slurmmentioning

confidence: 99%

Large-Scale Experiment for Topology-Aware Resource Management

Georgiou

Mercier

Villiermet

2018

Euro-Par 2017: Parallel Processing Workshops

View full text Add to dashboard Cite

A Resource and Job Management System (RJMS) is a crucial system software part of the HPC stack. It is responsible for efficiently delivering computing power to applications in supercomputing environments and its main intelligence relies on resource selection techniques to find the most adapted resources to schedule the users' jobs. In [8], we introduced a new topology-aware resource selection algorithm to determine the best choice among the available nodes of the platform based on their position in the network and on application behaviour (expressed as a communication matrix). We did integrate this algorithm as a plugin in Slurm and validated it with several optimization schemes by making comparisons with the default Slurm algorithm. This paper presents further experiments with regard to this selection process.

show abstract

Section: Slurmmentioning

confidence: 99%

Large-Scale Experiment for Topology-Aware Resource Management

Georgiou

Mercier

Villiermet

2018

Euro-Par 2017: Parallel Processing Workshops

View full text Add to dashboard Cite

show abstract

“…The algorithms within cons_res plugin are also scalable featuring best-t placement of jobs but they are more complex than select/linear since a ner granularity of allocatable resources is taken into account. One of the rst versions of the select/cons_res plugin is described in [1].…”

Section: Slurmmentioning

confidence: 99%

Topology-aware resource management for HPC applications

Georgiou

Jeannot

Mercier

et al. 2017

Proceedings of the 18th International Conference on Distributed Computing and Networking

View full text Add to dashboard Cite

The Resource and Job Management System (RJMS) is a crucial system software part of the HPC stack. It is responsible for eciently delivering computing power to applications in supercomputing environments. Its main intelligence relies on resource selection techniques to nd the most adapted resources to schedule the users' jobs. Improper resource selection operations may lead to poor performance executions and global system utilization along with increase of system fragmentation and jobs starvation. These phenomenas play a role in the increase of the platforms' total cost of ownership and should be minimized. This paper introduces a new topology-aware resource selection algorithm to determine the best choice among the available nodes of the platform based upon their position within the network and taking into account the applications communication matrix. To validate our approach, we integrated this algorithm as a plugin for Slurm, a popular and widespread HPC resource and job management system (RJMS). We validated our plugin with dierent optimization schemes by comparing with the default Slurm algorithm using both emulation of a large-scale platform, and by carrying out experiments in a real cluster.

show abstract

“…Job management was accomplished with SLURM. It is a resource manager that was first developed at LLNL and provides the basics needed to run a parallel job, namely, resource allocation, job executing/monitoring and queue managing of pending jobs, [17]- [19]. When coupled with MPI it provides a complete environment for parallel computing.…”

Section: Performance Configuration Setupmentioning

confidence: 99%

Parallelization of Diagnostics for Climate Model Development

McEnerney¹,

Ames²,

Christensen³

et al. 2016

JSEA

View full text Add to dashboard Cite

The parallelization of the diagnostics for climate research has been an important goal in the performance testing and improvement of the diagnostics for the Department of Energy's (DOE's) Accelerated Climate Modeling for Energy (ACME) project [1]. The primary mission of the ACME project is to build and test the next-generation Earth system model for current and future generations of computing systems operated by the DOE office of science computing facilities, including the envisioned exascale systems foreseen in the early part of the next decade. As part of the underpinning workflow environment, a diagnostics, model metrics, and intercomparison Python framework, called UVC Metrics was created to aid in testing and production execution of the model. This framework builds on common methods and similar metrics to accommodate and diagnose individual component models, such as atmosphere, land, ocean, sea ice, and land ice. This paper reports on initial parallelization of UVC Metrics for the atmosphere model component using two popular frameworks: MPI and SPARK. A timing study is presented to assess the performance of each method in which significant improvement was achieved for both frameworks despite I/O contentions with NFS. The advantages and disadvantages of each framework are also presented.

show abstract

Enhancing an Open Source Resource Manager with Multi-core/Multi-threaded Support

Cited by 13 publications

References 1 publication

Large-Scale Experiment for Topology-Aware Resource Management

Large-Scale Experiment for Topology-Aware Resource Management

Topology-aware resource management for HPC applications

Parallelization of Diagnostics for Climate Model Development

Contact Info

Product

Resources

About