Scheduling-Efficient Framework for Neural Network on Heterogeneous Distributed Systems and Mobile Edge Computing Systems

Abstract

As the volume of machine learning training data sets and the quantity of model parameters continue to grow, the pattern in which machine learning models are trained alone can no longer accommodate large-scale data environments. However, distributed systems and mobile edge computing systems are unpredictable and have heterogeneous nodes, resulting in interruptions in training or low convergence rate. In addition, existing distributed machine learning frameworks cannot guarantee a good convergence rate and speedup ratio in a variety of operating environments. Considering the above shortcomings, this paper proposes an adaptive scheduling framework for machine learning based on a heterogeneous distributed system and mobile edge computing system for machine learning model optimization. The framework detects and analyzes the dynamic changes of resources in the distributed system and mobile edge computing system through the resource detection system; then, the task scheduling system adaptively modifies the environmental parameters and schedules calculations. Relevant experiments conducted with the public data set show that the robustness and scalability of the framework are significantly better than the traditional distributed machine learning framework under the premise of ensuring high convergence rate. INDEX TERMS Heterogeneous distributed system, mobile edge computing system, adaptive scheduling, large-scale machine learning.