Minimizing Latency for Secure Coded Computing Using Secret Sharing via Staircase Codes

Abstract

We consider the setting of a Master server, M, who possesses confidential data (e.g., personal, genomic or medical data) and wants to run intensive computations on it, as part of a machine learning algorithm for example. The Master wants to distribute these computations to untrusted workers who have volunteered or are incentivized to help with this task. However, the data must be kept private (in an information theoretic sense) and not revealed to the individual workers. Some of the workers may be stragglers, e.g., slow or busy, and will take a random time to finish the task assigned to them. We are interested in reducing the delays experienced by the Master. We focus on linear computations as an essential operation in many iterative algorithms such as principal component analysis, support vector machines and other gradient-descent based algorithms. A classical solution is to use a linear secret sharing scheme, such as Shamir's scheme, to divide the data into secret shares on which the workers can perform linear computations. However, classical codes can provide straggler mitigation assuming a worst-case scenario of a fixed number of stragglers. We propose a solution based on new secure codes, called Staircase codes, introduced previously by two of the authors. Staircase codes allow flexibility in the number of stragglers up to a given maximum, and universally achieve the information theoretic limit on the download cost by the Master, leading to latency reduction. Under the shifted exponential model, we find upper and lower bounds on the Master's mean waiting time. We derive the distribution of the Master's waiting time, and its mean, for systems with up to two stragglers. For systems with any number of stragglers, we derive an expression that can give the exact distribution, and the mean, of the waiting time of the Master. We show that Staircase codes always outperform classical secret sharing codes. For instance, for codes with rate k/n = 1/2 Staircase codes can lead to up to 59% reduction in delay compared to classical secret sharing codes. We validate our results with extensive implementation on Amazon EC2 clusters.R. Bitar and S. El Rouayheb are with the ECE department of Rutgers University. P. Parag is with the ECE department of the Indian Institute of Science.In the previous example, even if there were no stragglers, M still has to wait for the full responses of two workers, and the response of the third one will not be used for decoding. In addition, M always has to decode Rx in order to decode Ax. Hence, more delays are incurred by spending communication and computation resources on decoding Rx, which is only needed for privacy. We overcome those limitations by using Staircase codes introduced in [12], [13] which do not always require decoding Rx. Thus, possibly reducing the computation load at the workers and the communication cost at the Master. In addition, Staircase codes allow more flexibility in the number of responses needed for decoding Ax, as explained in the next example.Example 2 (Staircase co...

Secure Distributed Computing With Straggling Servers Using Polynomial Codes

“…Several works have proposed securely outsourcing computation schemes with expensive homomorphic encryption [18], [19] or secret key generation [20]. Secure distributed computing schemes that address straggling effects have been proposed using secret sharing [21], [22] or staircase codes [7], [23] ensuring information-theoretic security on input data. However, these schemes assume that only one input of two matrices for multiplication is regarded as confidential data that must be preserved from workers, while our scheme using polynomial codes regards the two inputs as confidential data.…”

“…If the worker finishes its convolution task, it returnsc i to the 1) . (23) We note that the coefficients of x max{m,n}+m+n−1 , . .…”

Private and rateless adaptive coded matrix-vector multiplication

“…Recently there has been significant interest in applying coding theoretic techniques to speed up machine learning algorithms, as detailed in the next section. While most of the literature has focused on mitigation of slow workers, several recent works consider security on top of it, e.g., [10,[23][24][25][26]. In a recent work, the authors of [27] show that, under certain model assumptions, there are regimes, in terms of data splitting and number of workers used, where offloading tasks to the workers can be faster than doing the computations locally for large dimensional data.…”

“…Codes for privacy and straggler mitigation in distributed computing are first introduced in [3,26] where the authors consider a homogeneous setting and focus on matrixvector multiplication. The problem of private distributed matrix-matrix multiplication and private polynomial computation with straggler tolerance is studied [23,[55][56][57][58][59].…”

A Fundamental Storage-Communication Tradeoff for Distributed Computing With Straggling Nodes

“…(Thus, under the server-worker framework, all final outputs are calculated at the server and not at the distributed computing nodes as is the case in MapReduce systems.) The described server-worker framework with stragglers was treated, for example, in [15]- [25] with a focus on highdimensional matrix-by-matrix or matrix-by-vector multiplications and in [26]- [30] with a focus on gradient computing.…”