Optimal Task Allocation for Mobile Edge Learning with Global Training Time Constraints

Abstract

This paper proposes to minimize the loss of training a distributed machine learning (ML) model on nodes or learners connected via the resource-constrained wireless edge network by jointly optimizing the number of local and global updates and the task size allocation. The optimization is done while taking into account heterogeneous communication and computation capabilities of each learner. It is shown that the problem of interest cannot be solved analytically but by leveraging existing bounds on the difference between the optimal loss and the loss at any given iteration, an expression for the objective function is derived as a function of the number of local updates. It is shown that the problem is convex and can be solved by finding the argument that minimizes the loss. The result is then used to determine the batch sizes for each learner for the next global update step. The merits of the proposed solution, which is heterogeneity aware (HA), are exhibited by comparing its performance to the heterogeneity unaware (HU) approach.

Revisiting Communication-Efficient Federated Learning with Balanced Global and Local Updates

“…In our communication delay model, t i,cm will be treated as a constant. This treatment is common in works similar to this paper, such as [33] and [34]. Besides, similar to the analysis of the local training delay, we consider the communication delay of all users denoted as t cm = max{t i,cm }.…”

“…Besides the above challenges, the practical implementation of FL suffers from limited communication resources. The existing works can be generally classified into two categories: 1) minimizing the cost/budget to save communication resources [22]- [29], 2) minimizing the loss function subject to communication resources restriction [30]- [34]. In the first category, the convergence time in FL with optimal bandwidth allocation and scheduling strategies was studied in [22].…”

“…An adaptive FL algorithm to minimize the loss function with optimal number of local trainings and aggregations was proposed in [32]. This algorithm was extended in [33], [34] to obtain optimal number of scheduled users with resources limitation and total delay cost.…”

“…However, in most of existing works regarding the balanced global and local updates [32]- [34], there is a need to solve an optimization sub-problem according to the method proposed in [32] for each global aggregation, which will slow down the convergence. Besides, it is difficult, if not impossible, to obtain an accurate solution for the sub-problem, resulting in the deteriorated prediction accuracy.…”