A Fuzzy Neural Network Based Dynamic Data Allocation Model on Heterogeneous Multi-GPUs for Large-scale Computations

Abstract

The parallel computation capabilities of modern GPU (Graphics Processing Unit) processors have attracted increasing attention from researchers and engineers who have been conducting high computational throughput studies. However, current single GPU based engineering solutions are often struggle to fulfill their real-time requirements. Thus, the multi-GPU-based approach has become a popular and cost-effective choice for tackling the demands. In those cases, the computational load balancing over multiple GPU "nodes" is often the key and bottleneck that affect the quality and performance of the runtime system. The existing load balancing approaches are mainly based on the assumption that all GPU nodes in the same computer framework are of equal computational performance, which are often not the case due to cluster design and other legacy issues. This paper presents a novel dynamic load balancing (DLB) model for rapid data division and allocation on heterogeneous GPU nodes based on an innovative fuzzy neural network (FNN). In this research, a 5-state parameter feedback mechanism defining the overall cluster and node performances is proposed. The corresponding FNN-based DLB model will be capable of monitoring and predicting individual node performance under different workload scenarios. A real-time adaptive scheduler has been devised to reorganize the data inputs to each node when necessary to maintain their runtime computational performances. The devised model has been implemented on two dimensional (2D) discrete wavelet transform (DWT) tasks for evaluation. Experiment results show that this DLB model has enabled a high computational throughput while ensuring real-time and precision requirements from complex computational tasks.

A generic parallel computational framework of lifting wavelet transform for online engineering surface filtration

“…In addition, task-based load balancing schedulers that these approaches relied upon fall short to support applications with huge data throughputs but limited processing function(s) since there are very few "tasks" to schedule. To maximize the data parallelism in a heterogeneous multi-GPU system, this study has also devised an innovative data-oriented dynamic load balancing (DLB) model based on an improved FNN for rapid measured data division and allocation on heterogeneous GPU nodes [38].…”

“…LWT applications are data-oriented computations, i.e., huge data processing but limited CUDA kernels. Thus, this study devised a data-oriented dynamic load balancing (DLB) model for rapid and dynamic dataset division and allocation on heterogeneous GPU nodes by using a fuzzy neural network (FNN) framework [38]. The data-oriented DLB model can minimize the overall processing time by dynamically adjusting the number of data subsets in a group for each GPU node according to runtime feedbacks.…”

“…The overall framework of the FNN based DLB model devised in the previous work is illustrated in Fig. 18 [39] [38]. In this model, five state feedback parameters (fuzzy sets), i.e., floating-point operation performance (F), global memory size (M), parallel ability (P), the occupancy rate of computational resources of a GPU (UF) and the occupancy rate of the global memory space of a GPU (UM), relating closely to the overall computational performance are defined.…”

“…20 shows the curve of the improved softplus.Based on the three new state feedback parameters and the improved softplus, this study constructed a more effective DLB model that integrates the fuzzy theory, artificial neural networks (ANN) and the back propagation algorithm. The comparison experiment between the improve DLB model and the conventional DLB model in[39] [38] is presented in Section 6.2 (the fourth fifth experiment).…”

HALCONE : A Hardware-Level Timestamp-based Cache Coherence Scheme for Multi-GPU systems

“…For instance, Goyal et al [12] trained ResNet-50 in only 1 hour using 256 GPUs, which would have otherwise taken more than a week using a single GPU. MGPU systems are also commonly used for parallelizing irregular graph applications [5,28,36,37] and facilitating large-scale simulations in different domains including physics [41], computational algebra [38], surface metrology [40] and medicine [13]. GPU applications are evolving to support ever-larger datasets and demand data communications not only within a GPU but also across multiple GPUs in the system.…”