TiFL: A Tier-based Federated Learning System

Chai, Zheng; Ali, Ahsan; Zawad, Syed; Truex, Stacey; Anwar, Ali; Baracaldo, Nathalie; Zhou, Yi; Ludwig, Heiko; Yan, Feng; Cheng, Yue

doi:10.1145/3369583.3392686

Cited by 202 publications

(104 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several popular platforms for FL, which have been summarized in Table 3 in terms of their focus and supporting software. There are plenty of platforms and architectures for FL, with more examples including [53], [54], [55], [56]. These platforms and architectures help refine FL better.…”

Section: Figure 10 Framework Of Fadlmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

This paper provides a comprehensive study of Federated Learning (FL) with an emphasis on enabling software and hardware platforms, protocols, real-life applications and use-cases. FL can be applicable to multiple domains but applying it to different industries has its own set of obstacles. FL is known as collaborative learning, where algorithm(s) get trained across multiple devices or servers with decentralized data samples without having to exchange the actual data. This approach is radically different from other more established techniques such as getting the data samples uploaded to servers or having data in some form of distributed infrastructure. FL on the other hand generates more robust models without sharing data, leading to privacy-preserved solutions with higher security and access privileges to data. This paper starts by providing an overview of FL. Then, it gives an overview of technical details that pertain to FL enabling technologies, protocols, and applications. Compared to other survey papers in the field, our objective is to provide a more thorough summary of the most relevant protocols, platforms, and real-life use-cases of FL to enable data scientists to build better privacy-preserving solutions for industries in critical need of FL. We also provide an overview of key challenges presented in the recent literature and provide a summary of related research work. Moreover, we explore both the challenges and advantages of FL and present detailed service use-cases to illustrate how different architectures and protocols that use FL can fit together to deliver desired results.

show abstract

Section: Figure 10 Framework Of Fadlmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The row-wise partitioning of weight matrices induces neuron partitioning in each layer so that all computations related to a neuron are performed by a single processor. As shown in (8) and (9), to perform W k m x k −1 , processor P m needs to receive all x k −1 -vector rows corresponding to column indices in cols(W k m ). It is important to note that vectors x k−1 mn andx k −1 nm are placeholders that keep coordinates of nonzero entries.…”

Section: Parallel Sparse Feedforwardmentioning

confidence: 99%

Partitioning sparse deep neural networks for scalable training and inference

Demirci

Ferhatosmanoğlu

2021

Proceedings of the ACM International Conference on Supercomputing

View full text Add to dashboard Cite

The state-of-the-art deep neural networks (DNNs) have significant computational and data management requirements. The size of both training data and models continue to increase. Sparsification and pruning methods are shown to be effective in removing a large fraction of connections in DNNs. The resulting sparse networks present unique challenges to further improve the computational efficiency of training and inference in deep learning. Both the feedforward (inference) and backpropagation steps in stochastic gradient descent (SGD) algorithm for training sparse DNNs involve consecutive sparse matrix-vector multiplications (SpMVs). We first introduce a distributed-memory parallel SpMV-based solution for the SGD algorithm to improve its scalability. The parallelization approach is based on row-wise partitioning of weight matrices that represent neuron connections between consecutive layers. We then propose a novel hypergraph model for partitioning weight matrices to reduce the total communication volume and ensure computational load-balance among processors. Experiments performed on sparse DNNs demonstrate that the proposed solution is highly efficient and scalable. By utilizing the proposed matrix partitioning scheme, the performance of our solution is further improved significantly.

show abstract

“…Despite fruitful research results on FL regarding advanced learning algorithms [7,8], data heterogeneity [9,10], personalization [11][12][13][14], fairness [15,16], system design [17][18][19] and privacypreserving frameworks [20][21][22] etc., there are only a few focusing on hyper-parameters tuning for FL [23,4,[24][25][26]3]. In [23], Dai et.al.…”

Section: Related Workmentioning

confidence: 99%

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

Zhou¹,

Ram²,

Salonidis³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO). We introduce Federated Loss SuRface Aggregation (FLoRA), the first FL-HPO solution framework that can address use cases of tabular data and gradient boosting training algorithms in addition to stochastic gradient descent/neural networks commonly addressed in the FL literature. The framework enables single-shot FL-HPO, by first identifying a good set of hyper-parameters that are used in a single FL training. Thus, it enables FL-HPO solutions with minimal additional communication overhead compared to FL training without HPO. Our empirical evaluation of FLoRA for Gradient Boosted Decision Trees on seven OpenML data sets demonstrates significant model accuracy improvements over the considered baseline, and robustness to increasing number of parties involved in FL-HPO training.

show abstract

TiFL: A Tier-based Federated Learning System

Cited by 202 publications

References 10 publications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Partitioning sparse deep neural networks for scalable training and inference

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

Contact Info

Product

Resources

About