Malleable parallelism with minimal effort for maximal throughput and maximal hardware load

Abstract

In practice, standard scheduling of parallel computing jobs almost always leaves significant portions of the available hardware unused, even with many jobs still waiting in the queue. The simple reason is that the resource requests of these waiting jobs are fixed and do not match the available, unused resources. However, with alternative but existing and well-established techniques it is possible to achieve a fully automated, adaptive parallelism that does not need pre-set, fixed resources. Here, we demonstrate that such an adaptively parallel program can run productively on a machine that is traditionally considered "full" and thus can indeed fill in all such scheduling gaps, even in real-life situations on large supercomputers in which a fixed-size job could not have started. keywords adaptive parallelism, malleable parallelism, scheduling, genetic algorithms, non-deterministic global optimization

An study of the effect of process malleability in the energy efficiency on GPU-based clusters

“…Although process malleability has been integrated into different types of applications such as: master-worker [4], non-iterative [10] or benchmarks [17], the main target of malleability are iterative applications [7] since they present clear processes synchronization points where job reconfigurations can be easily triggered [5,11,16,26]. Malleability has been also implemented with different approaches and frameworks, such as: non-standard MPI with ULFM [14], Checkpoint-restart [6], CHARM++ [8], Java virtual machine [25], etc. Concretely, this research relies on the dynamic management of resources (DMR) process malleability framework [12], a standard MPI-based solution which provides a modular design that allows its integration with other programming models, such as CUDA.…”