Collaborative Decoding of Polynomial Codes for Distributed Computation

Subramaniam, Adarsh M.; Heidarzadeh, Anoosheh; Narayanan, Krishna R.

doi:10.1109/itw44776.2019.8989254

Cited by 16 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such ideas have also been extended to also provide Byzantine robustness and data privacy. Specifically, the coded computing paradigm has recently emerged by adapting erasure coding ideas to design straggler-resilient, Byzantine-robust and private distributed computing systems often involving polynomial-type computations (Yu et al 2019;Subramaniam, Heidarzadeh, and Narayanan 2019;Tang et al 2021;So, Guler, and Avestimehr 2020;Sohn et al 2020;So et al 2021). However, many applications involve non-polynomial computations as training and inference of neural networks.…”

Section: Related Workmentioning

confidence: 99%

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

Soleymani

Ali

Mahdavifar

et al. 2022

AAAI

View full text Add to dashboard Cite

Due to the surge of cloud-assisted AI services, the problem of designing resilient prediction serving systems that can effectively cope with stragglers and minimize response delays has attracted much interest. The common approach for tackling this problem is replication which assigns the same prediction task to multiple workers. This approach, however, is inefficient and incurs significant resource overheads. Hence, a learning-based approach known as parity model (ParM) has been recently proposed which learns models that can generate ``parities’’ for a group of predictions to reconstruct the predictions of the slow/failed workers. While this learning-based approach is more resource-efficient than replication, it is tailored to the specific model hosted by the cloud and is particularly suitable for a small number of queries (typically less than four) and tolerating very few stragglers (mostly one). Moreover, ParM does not handle Byzantine adversarial workers. We propose a different approach, named Approximate Coded Inference (ApproxIFER), that does not require training any parity models, hence it is agnostic to the model hosted by the cloud and can be readily applied to different data domains and model architectures. Compared with earlier works, ApproxIFER can handle a general number of stragglers and scales significantly better with the number of queries. Furthermore, ApproxIFER is robust against Byzantine workers. Our extensive experiments on a large number of datasets and model architectures show significant degraded mode accuracy improvement by up to 58% over ParM.

show abstract

Section: Related Workmentioning

confidence: 99%

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

Soleymani

Ali

Mahdavifar

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…In fact, this is a limitation of many other problems such as verifiable computing and machine learning [212], [213]. Recently, several works have extended LCC to the analog domain to address these challenges [210], [211], [214], [215], but they either focus only on straggler-mitigation [215], privacy [210], [211] or Byzantine-robustness [214]. An interesting open problem is to design a framework that jointly tackles these three challenges in the analog domain.…”

Section: Secure Model Aggregation In Federated Learningmentioning

confidence: 99%

Private Retrieval, Computing and Learning: Recent Progress and Future Challenges

Ulukuş¹,

Avestimehr²,

Gastpar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most of our lives are conducted in the cyberspace. The human notion of privacy translates into a cyber notion of privacy on many functions that take place in the cyberspace. This article focuses on three such functions: how to privately retrieve information from cyberspace (privacy in information retrieval), how to privately leverage large-scale distributed/parallel processing (privacy in distributed computing), and how to learn/train machine learning models from private data spread across multiple users (privacy in distributed (federated) learning). The article motivates each privacy setting, describes the problem formulation, summarizes breakthrough results in the history of each problem, and gives recent results and discusses some of the major ideas that emerged in each field. In addition, the crosscutting techniques and interconnections between the three topics are discussed along with a set of open problems and challenges. I. INTRODUCTIONPrivacy is an important part of human life. This article considers privacy in the context of three distinct but related engineering applications, namely, privacy in retrieving information, privacy in computing functions, and privacy in learning. In the first sub-topic of private information retrieval, a user wishes to download a content from publicly accessible databases in such a way that the databases do not learn which particular content the user has downloaded. Towards that goal, the user creates ambiguity by its actions during the download. This strategy prevents databases from guessing which content the user has downloaded. This in turn preserves the user's privacy because what is downloaded leaks information about interest and intent on the part of the user. In the second sub-topic of private computing, a user wishes to compute a function but

show abstract

“…The master can thus use this fact to alleviate the assumption of limited knowledge of the malicious workers. In [43] and [44] variants of Reed-Solomon codes are used to half the damage of the malicious workers when the workers introduce random noise and when the workers introduce any kind of noise, respectively. The disadvantage of [44] is the high computational complexity incurred by the master.…”

Section: Introductionmentioning

confidence: 99%

“…The ideas used in [50] are similar to the ideas used in this work. However, [28], [41], [43], [44] and [50] consider security in settings where the workers are assumed to have similar resources. In addition, a maximum amount of stragglers is assumed.…”

Section: Introductionmentioning

confidence: 99%

Secure Private and Adaptive Matrix Multiplication Beyond the Singleton Bound

Hofmeister¹,

Bitar²,

Xhemrishi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Consider the problem of designing secure and private codes for distributed matrix-matrix multiplication. A master server owns two private matrices A and B and hires worker nodes to help computing their multiplication. The matrices should remain information-theoretically private from the workers. Some of the workers are malicious and return corrupted results to the master. This work is motivated by the literature on myopic adversaries in network coding and distributed storage. Security beyond the Singleton bound is possible when the adversary has limited knowledge about the master's data and probabilistic decoding is acceptable. The key observation in this setting is that the master is the sender and the receiver. Therefore, the master enjoys a plethora of advantages that enable coding for security beyond the Singleton bound.We design a framework for security against malicious adversaries in private matrix-matrix multiplication. Our main goal is to apply this security framework to schemes with adaptive rates previously introduced by a subset of the authors. Adaptive schemes divide the workers into clusters and thus provide flexibility in trading decoding complexity for efficiency. Checking the integrity of the computation per cluster has low complexity but costs deleting the results of a whole cluster with at least one malicious worker. Checking the integrity of the results per worker is more complex but allows an efficient use of the non-malicious workers. Our scheme, called SRPM3, provides a computationally efficient security check that detects malicious workers with high probability and can tolerate the presence of an arbitrary number of malicious workers. We provide simulation results that validate our theoretical findings.

show abstract

Collaborative Decoding of Polynomial Codes for Distributed Computation

Cited by 16 publications

References 16 publications

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

Private Retrieval, Computing and Learning: Recent Progress and Future Challenges

Secure Private and Adaptive Matrix Multiplication Beyond the Singleton Bound

Contact Info

Product

Resources

About