Bivariate Polynomial Coding for Straggler Exploitation with Heterogeneous Workers

Abstract

Polynomial coding has been proposed as a solution to the straggler mitigation problem in distributed matrix multiplication. Previous works employ univariate polynomials to encode matrix partitions. Such schemes greatly improve the speed of distributed computing systems by making the task completion time to depend only on the fastest workers. However, they completely ignore the work done by the slowest workers resulting in inefficient use of computing resources. In order to exploit the partial computations of the slower workers, we further decompose the overall matrix multiplication task into even smaller subtasks, and we propose bivariate polynomial codes. We show that these codes are a more natural choice to accommodate the additional decomposition of subtasks, and to exploit the heterogeneous storage and computation resources at workers. However, in contrast to univariate polynomial decoding, guarantying decodability with multivariate interpolation is much harder. We propose two bivariate polynomial coding schemes and study their decodability conditions. Our numerical results show that bivariate polynomial coding considerably reduces the computation time of distributed matrix multiplication.

Bivariate Polynomial Codes for Secure Distributed Matrix Multiplication

“…Our coding scheme is based on bivariate polynomial codes [10], [11]. Compared to their univariate counterparts, bivariate polynomial codes allow workers to complete more sub-tasks under the same upload cost budget, which, in turn, improves the average computation time and helps to satisfy the privacy requirements.…”

“…Thus, the total upload cost of the SBP coding scheme is N (rs/K + mst/L) log 2 (q) bits. The upload cost could be reduced with a direct extension of the bivariate schemes presented in [10], [11], in which m computations are assigned to a worker by sending as few as √ m coded partitions of A and B. However, the direct extension of this scheme to the secure computation case results in a large recovery threshold.…”

“…Moreover, the multi-message approach makes finishing the task possible even if there are not as many available workers as the recovery threshold. However, as discussed in [10], direct extension of polynomial-type codes to the multi-message setting by simply assigning multiple sub-products to the workers increase the upload communication costs, which is defined as the number of bits sent from the master to each worker, or equivalently, the storage required per worker. To combat this effect, product codes are proposed in [7] for the multi-message distributed matrix multiplication problem.…”

“…However, with product codes, sub-products are no longer fully one-to-any replaceable, which degrades their performance. The bivariate polynomial codes are introduced in [10], [11] to address this issue, achieving a better trade-off between the upload cost and average computation time.…”

Securely Straggler-Exploiting Coded Computation for Distributed Matrix Multiplication

“…In addition, the authors in [14] have proposed a new fine-grained task allocation scheme to efficiently utilize the straggling workers' computation ability to speed up the entire task execution. Furthermore, recent papers [15]- [17] have suggested new encoding schemes for matrix multiplication to efficiently utilize the memory of the workers and to lower the communication load in distributed computing.…”