Cooperative Distributed GPU Power Capping for Deep Learning Clusters

Abstract

The recent GPU-based clusters that handle deep learning (DL) tasks have the features of GPU device heterogeneity, a variety of deep neural network (DNN) models, and high computational complexity. Thus, the traditional power capping methods for CPU-based clusters or small-scale GPU devices do not apply to the GPU-based clusters handling DL tasks. This paper develops a cooperative distributed GPU power capping (CD-GPC) system for GPU-based clusters, aiming to minimize the training completion time of invoked DL tasks without exceeding the limited power budget. Specifically, we first design the frequency scaling (FS) approach using the online model estimation based on the recursive least square (RLS) method. This approach achieves the accurate tuning for DL task training time and power usage of GPU devices without needing offline profiling. Then, we formulate the proposed FS problem as a Lagrangian dual decomposition-based economic model predictive control (EMPC) problem for large-scale heterogeneous GPU clusters. We conduct both the NVIDIA GPU-based lab-scale real experiments and real job trace-based simulation experiments for performance evaluation. Experimental results validate that the proposed system improves the power capping accuracy to have a mean absolute error ă 1%, and reduces the deadline violation ratio of invoked DL tasks by 21.5% compared with other recent counterparts.

Energy-Efficient and Timeliness-Aware Continual Learning Management System

“…The continuous advancement of GPU architectures enhances the speed of DNN model training but also results in significantly higher energy consumption. Despite improvements in manufacturing processes, GPU devices continue to exhibit high absolute energy usage [18]. It is worth noting that even with the significant increase in energy usage, we may only observe marginal improvements in DNN model training performance, which depend on the specific DNN model types and characteristics of worker nodes.…”

Renewable-Aware Frequency Scaling Approach for Energy-Efficient Deep Learning Clusters

“…Now, we present a GPU core frequency-based performance model for DL jobs, utilizing a statistical modeling approach. This model is grounded in the relationship t ∝ 1 f (time is inversely proportional to frequency), where t represents the DL job processing time and f denotes the frequency value, as discussed in [17,18]. In this model, λ F i and λ B i are the performance model coefficients for feed-forward and back-propagation processes in DNN model training jobs, respectively.…”

Offline performance and energy consumption prediction model of deep learning training tasks

“…4.1.2 Model selection. According to some previous works 1,17 , we adopt some popular and classical DNN models as the evaluation models, including AlexNet, VGG11, VGG16, ResNet18, ResNet50 and DenseNet121.…”