A Scalable Framework for Online Power Modelling of High-Performance Computing Nodes in Production

Abstract

Power and thermal design and management are critical components of high performance computing (HPC) systems, due to their cutting-edge position in terms of high power density and large total power consumption. Many HPC power management strategies rely on the availability of accurate compact power models, capable of predicting power consumption and tracking its sensitivity to workload parameters and operating points. In this paper we describe a methodology and a framework for training two of the best-in-class power models directly on the online in production nodes and without requiring dedicated training instances. The compact power models are obtained using an online regression-based approach which can track non-stationary workloads and hardware variability. Our experiments on a real-life HPC system demonstrate that the models achieve very high accuracy over all operating modes. We also demonstrate the scalability of our approach and the small amount of resources needed for the online modeling, for both the training and inference phases.

Robust Identification of Thermal Models for In-Production High-Performance-Computing Clusters With Machine Learning-Based Data Selection

“…By comparing Fig. 2 with the results in [27], we note that the new model performs slightly worse than the old one, as expected because we use significantly less features and we are estimating also the individual cores contributions. However the additional error is not very significant, and anyways for more than 90% of the points the error in package power is below 10W, which corresponds to roughly 1W error per core (assuming uniform distribution of the power) and about 10% of the maximum power.…”

“…In order to meet all these requirements, we have decided to extend the distributed and scalable monitoring framework that we presented in [26], [27]. As far as the thermal model is concerned, we have used a variation to the algorithm presented in [23], which makes use of an identification algorithm (see Sec.…”

“…The power model is based on the measurement of appropriate metrics derived from the performance counters, as explained in [27]. To choose a subset of the metrics, among all the available ones, we have calculated Pearson's correlation of the metrics with the measured package power.…”

“…With this formulation we can easily extract the partial powers, i.e., P unc is an estimate of the power dissipated by the uncore and P k is the power dissipated by core k. Note that also for the derivation of the power model we use real workloads, and this poses the same challenges we discussed above and in [27]. However, since the accuracy of the power model is not critical and the model itself is relatively simple, we have verified that we are always able to derive a model with relatively good stability and accuracy if we use a long enough interval for the training data, in our case generally 3 days as in [27]. Moreover, as we showed in [27], the training of the power model and its evaluation can be done very efficiently in our scalable framework and they impose only a minimal overhead.…”

“…An overview of the model's performance is reported in Fig. 2, which shows the percentage of points where the error between the estimated and the measured power is below a certain threshold (3.23W and 9.68W, equivalent to the 1 • C and 3 • C errors we used in [27]). Each point on the x-axis is a different training time, and for each training time and threshold they are shown the median relative number of points below the threshold together with the 25% and 75% percentiles (the error bars).…”