Adaptive Verifiable Coded Computing: Towards Fast, Secure and Private Distributed Machine Learning

Tang, Tingting; Ali, Ramy E.; Hashemi, Hanieh; Gangwani, Tynan; Avestimehr, Salman; Annavaram, Murali

doi:10.48550/arxiv.2107.12958

Cited by 4 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The disadvantage of [44] is the high computational complexity incurred by the master. Concurrently and independently of our work, the authors of [50] present a scheme to tolerate malicious workers in coded computing with low computational complexity. The ideas used in [50] are similar to the ideas used in this work.…”

Section: Introductionmentioning

confidence: 98%

“…Concurrently and independently of our work, the authors of [50] present a scheme to tolerate malicious workers in coded computing with low computational complexity. The ideas used in [50] are similar to the ideas used in this work. However, [28], [41], [43], [44] and [50] consider security in settings where the workers are assumed to have similar resources.…”

Section: Introductionmentioning

confidence: 98%

“…The ideas used in [50] are similar to the ideas used in this work. However, [28], [41], [43], [44] and [50] consider security in settings where the workers are assumed to have similar resources. In addition, a maximum amount of stragglers is assumed.…”

Section: Introductionmentioning

confidence: 99%

“…In this work we are interested in coded computing for heterogeneous environments where the number of stragglers can change over time. Our scheme enjoys a smaller computational complexity than that of [50] which can be adaptively increased to increase the efficiency of the scheme as will be explained in the sequel. In [42] secure matrix-vector multiplication in heterogeneous environments is ensured by using homomorphic hash functions.…”

Section: Introductionmentioning

confidence: 99%

“…The master can thus verify with high probability whether the computation returned from each cluster is corrupted. Verifying the computation per clusters results in a more efficient verification than the scheme presented in [50]. As an extra layer of verification, if the computation of a certain cluster of workers is corrupted, the master can run Freivald's algorithm on the computation of each worker to detect, with high probability, the malicious workers and use the results of the honest workers.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Secure Private and Adaptive Matrix Multiplication Beyond the Singleton Bound

Hofmeister¹,

Bitar²,

Xhemrishi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Consider the problem of designing secure and private codes for distributed matrix-matrix multiplication. A master server owns two private matrices A and B and hires worker nodes to help computing their multiplication. The matrices should remain information-theoretically private from the workers. Some of the workers are malicious and return corrupted results to the master. This work is motivated by the literature on myopic adversaries in network coding and distributed storage. Security beyond the Singleton bound is possible when the adversary has limited knowledge about the master's data and probabilistic decoding is acceptable. The key observation in this setting is that the master is the sender and the receiver. Therefore, the master enjoys a plethora of advantages that enable coding for security beyond the Singleton bound.We design a framework for security against malicious adversaries in private matrix-matrix multiplication. Our main goal is to apply this security framework to schemes with adaptive rates previously introduced by a subset of the authors. Adaptive schemes divide the workers into clusters and thus provide flexibility in trading decoding complexity for efficiency. Checking the integrity of the computation per cluster has low complexity but costs deleting the results of a whole cluster with at least one malicious worker. Checking the integrity of the results per worker is more complex but allows an efficient use of the non-malicious workers. Our scheme, called SRPM3, provides a computationally efficient security check that detects malicious workers with high probability and can tolerate the presence of an arbitrary number of malicious workers. We provide simulation results that validate our theoretical findings.

show abstract

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Secure Private and Adaptive Matrix Multiplication Beyond the Singleton Bound

Hofmeister¹,

Bitar²,

Xhemrishi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware

Hashemi

Wang

Annavaram

2021

MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

Self Cite

View full text Add to dashboard Cite

Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train or infer with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. Cloud systems are vulnerable to attackers that compromise the privacy of data and integrity of computations. Tackling such a challenge requires unifying theoretical privacy algorithms with hardware security capabilities. This paper presents DarKnight, a framework for large DNN training while protecting input privacy and computation integrity. DarKnight relies on cooperative execution between trusted execution environments (TEE) and accelerators, where the TEE provides privacy and integrity verification, while accelerators perform the bulk of the linear algebraic computation to optimize the performance. In particular, DarKnight uses a customized data encoding strategy based on matrix masking to create input obfuscation within a TEE. The obfuscated data is then offloaded to GPUs for fast linear algebraic computation. DarKnight's data obfuscation strategy provides provable data privacy and computation integrity in the cloud servers. While prior works tackle inference privacy and cannot be utilized for training, DarKnight's encoding scheme is designed to support both training and inference.

show abstract

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

Soleymani

Ali

Mahdavifar

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

Due to the surge of cloud-assisted AI services, the problem of designing resilient prediction serving systems that can effectively cope with stragglers and minimize response delays has attracted much interest. The common approach for tackling this problem is replication which assigns the same prediction task to multiple workers. This approach, however, is inefficient and incurs significant resource overheads. Hence, a learning-based approach known as parity model (ParM) has been recently proposed which learns models that can generate ``parities’’ for a group of predictions to reconstruct the predictions of the slow/failed workers. While this learning-based approach is more resource-efficient than replication, it is tailored to the specific model hosted by the cloud and is particularly suitable for a small number of queries (typically less than four) and tolerating very few stragglers (mostly one). Moreover, ParM does not handle Byzantine adversarial workers. We propose a different approach, named Approximate Coded Inference (ApproxIFER), that does not require training any parity models, hence it is agnostic to the model hosted by the cloud and can be readily applied to different data domains and model architectures. Compared with earlier works, ApproxIFER can handle a general number of stragglers and scales significantly better with the number of queries. Furthermore, ApproxIFER is robust against Byzantine workers. Our extensive experiments on a large number of datasets and model architectures show significant degraded mode accuracy improvement by up to 58% over ParM.

show abstract

Adaptive Verifiable Coded Computing: Towards Fast, Secure and Private Distributed Machine Learning

Cited by 4 publications

References 12 publications

Secure Private and Adaptive Matrix Multiplication Beyond the Singleton Bound

Secure Private and Adaptive Matrix Multiplication Beyond the Singleton Bound

DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

Contact Info

Product

Resources

About