A Survey of Coded Distributed Computing

Abstract

Distributed computing has become a common approach for large-scale computation of tasks due to benefits such as high reliability, scalability, computation speed, and costeffectiveness. However, distributed computing faces critical issues related to communication load and straggler effects. In particular, computing nodes need to exchange intermediate results with each other in order to calculate the final result, and this significantly increases communication overheads. Furthermore, a distributed computing network may include straggling nodes that run intermittently slower. This results in a longer overall time needed to execute the computation tasks, thereby limiting the performance of distributed computing. To address these issues, coded distributed computing (CDC), i.e., a combination of coding theoretic techniques and distributed computing, has been recently proposed as a promising solution. Coding theoretic techniques have proved effective in WiFi and cellular systems to deal with channel noise. Therefore, CDC may significantly reduce communication load, alleviate the effects of stragglers, provide fault-tolerance, privacy and security. In this survey, we first introduce the fundamentals of CDC, followed by basic CDC schemes. Then, we review and analyze a number of CDC approaches proposed to reduce the communication costs, mitigate the straggler effects, and guarantee privacy and security. Furthermore, we present and discuss applications of CDC in modern computer networks. Finally, we highlight important challenges and promising research directions related to CDC.

Data Monitoring based Power Network Investment Optimization and Empirical Research: Survey and Prospect

“…Secondly, distributed computing demonstrates high computation speed as the computational load is shared among various computing nodes. Thirdly, these systems are scalable, meaning that adding more computing nodes enhances the corresponding computing capacity [46]. Given these advantages, distributed computing has found application in numerous real-life scenarios such as distributed machine learning systems [47], and blockchain systems [48].…”

Multi-User Linearly Separable Computation: A Coding Theoretic Approach

“…Such works can be found in the context of computational accuracy [6]- [9], latency and straggler mitigation [10]- [13], scalability [14], [15], security and privacy [16]- [20], as well as in the context of communication and computation complexity [21]- [27]. For a detailed survey of various such works, the reader is encouraged to see [28], [29].…”

Multi-User Linearly-Separable Distributed Computing

“…Distributed computing naturally entails several challenges that involve accuracy [3]- [5], scalability [6]- [10], privacy and security [11]- [23], as well as latency and straggler mitigation [24]- [31]. For a detailed survey of related research works, the reader is referred to [32], [33].…”