Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy

Abstract

We consider a scenario involving computations over a massive dataset stored distributedly across multiple workers, which is at the core of distributed learning algorithms. We propose Lagrange Coded Computing (LCC), a new framework to simultaneously provide (1) resiliency against stragglers that may prolong computations; (2) security against Byzantine (or malicious) workers that deliberately modify the computation for their benefit; and (3) (information-theoretic) privacy of the dataset amidst possible collusion of workers. LCC, which leverages the well-known Lagrange polynomial to create computation redundancy in a novel coded form across workers, can be applied to any computation scenario in which the function of interest is an arbitrary multivariate polynomial of the input dataset, hence covering many computations of interest in machine learning. LCC significantly generalizes prior works to go beyond linear computations. It also enables secure and private computing in distributed settings, improving the computation and communication efficiency of the state-of-the-art. Furthermore, we prove the optimality of LCC by showing that it achieves the optimal tradeoff between resiliency, security, and privacy, i.e., in terms of tolerating the maximum number of stragglers and adversaries, and providing data privacy against the maximum number of colluding workers. Finally, we show via experiments on Amazon EC2 that LCC speeds up the conventional uncoded implementation of distributed least-squares linear regression by up to 13.43×, and also achieves a 2.36×-12.65× speedup over the state-of-the-art straggler mitigation strategies.

Multi-Party Proof Generation in QAP-based zk-SNARKs

“…The sharing must be such that if any subset of T servers collude, they gain no information about the input data. Various approaches, such as ramp sharing [61] and Lagrange sharing [51] have been proposed for such sharing. In this paper, we use Lagrange Sharing [51], which works as follows:…”

“…Various approaches, such as ramp sharing [61] and Lagrange sharing [51] have been proposed for such sharing. In this paper, we use Lagrange Sharing [51], which works as follows:…”

“…To offload computation task on private data to some external untrusted nodes, a fundamental approach is multi party computation. The multi party computation was initially introduced in the early 1980s by Yao [47], and has been followed in many works [46], [48]- [51]. For a survey about multi party computation see [52].…”

Numerically Stable Binary Gradient Coding

“…If (s + 1) n, the worst case occurs when all workers corresponding to blocks of B C1 and − 1 blocks of B C2 respond, along with only one worker from either of the two remaining blocks. By (11), the total number of responsive workers is n − s. In the best case, we need a single worker corresponding to each block of B C2 to respond, i.e., = n s+1 responsive workers. Similarly, if (s + 1) | n, in the best case we require = n s+1 workers to respond, and in the worst case n − ( − 1) • (s + 1) + 1 = n − s many workers.…”

PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously

“…PolyShard is inspired by recent developments in coded computing [12]- [20], in particular Lagrange Coded Computing [20], which provides a transformative framework for injecting computation redundancy in unorthodox coded forms in order to deal with failures and errors in distributed computing. The key idea behind PolyShard is that instead of storing and processing a single uncoded shard as in convention, each node stores and computes on a coded shard of the same size that is generated by linearly mixing uncoded shards, using the well-known Lagrange polynomial.…”