Eco-Efficient Resource Management in HPC Clusters through Computer Intelligence Techniques

Abstract

High Performance Computing Clusters (HPCCs) are common platforms for solving both up-to-date challenges and high-dimensional problems faced by IT service providers. Nonetheless, the use of HPCCs carries a substantial and growing economic and environmental impact, owing to the large amount of energy they need to operate. In this paper, a two-stage holistic optimisation mechanism is proposed to manage HPCCs in an eco-efficiently manner. The first stage logically optimises the resources of the HPCC through reactive and proactive strategies, while the second stage optimises hardware allocation by leveraging a genetic fuzzy system tailored to the underlying equipment. The model finds optimal trade-offs among quality of service, direct/indirect operating costs, and environmental impact, through multiobjective evolutionary algorithms meeting the preferences of the administrator. Experimentation was done using both actual workloads from the Scientific Modelling Cluster of the University of Oviedo and synthetically-generated workloads, showing statistical evidence supporting the adoption of the new mechanism.

Improving utilization of heterogeneous clusters

“…A different approach to the scheduling problem is to use a knowledge-based system (KBS) comprised of an individual set of if-then rules that depend on certain parameters [21][22][23]. In this way [21] presents a hybrid genetic fuzzy system (HGFS) that combines both a fuzzy and a non-fuzzy sets of rules.…”

“…On the other hand, in [22], through a forecast of the future workload and according to a utility function, an optimization problem is solved. Finally, in [23], a two-stage holistic optimization mechanism is proposed, composed of a stage that logically optimizes the resources and another that optimizes hardware allocation by leveraging a genetic fuzzy system. The model finds optimal trade-offs among different objectives.…”