Constraint Programming-Based Job Dispatching for Modern HPC Applications

Abstract

Constraint Programming (CP) is a wellestablished area in AI as a programming paradigm for modelling and solving discrete optimization problems, and it has been been successfully applied to tackle the on-line job dispatching problem in HPC systems including those running modern applications. The limitations of the available CP-based job dispatchers may hinder their practical use in today's systems that are becoming larger in size and more demanding in resource allocation. In an attempt to bring basic AI research closer to a deployed application, we present a new CP-based on-line job dispatcher for modern HPC systems and applications. Unlike its predecessors, our new dispatcher tackles the entire problem in CP and its model size is independent of the system size. Experimental results based on a simulation study show that with our approach dispatching performance increases significantly in a large system and in a system where allocation is nontrivial.

Parallel Empirical Evaluations: Resilience despite Concurrency

“…Our overall approach fits well into standard highperformance computing (HPC) environments, which encourages the use of modern cluster environments that was previously seen highly problematic for replicability. (Bridi 2018;Galleguillos et al 2019). The influence of hardware efficiency and measurements is well established (Georgiou et al 2014;Beyer, Löwe, and Wendler 2019;Vercellino et al 2023).…”