BaPa: A Novel Approach of Improving Load Balance in Parallel Matrix Factorization for Recommender Systems

Guo, Ruixin; Zhang, Feng; Wang, Lizhe; Zhang, Wusheng; Lei, Xinya; Ranjan, Rajiv; Zomaya, Albert Y.

doi:10.1109/tc.2020.2997051

Cited by 6 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DSGD, DSGD++ and NOMAD has the same total communication volume during an SGD epoch per processor which is equals to F × M × K as discussed in Section 3.1. The number of messages sent per processor during an q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Amz Books Amz q DSGD P2P H&C (a) F = 16 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Amz Books Amz [15] proposed a novel framework, BaPa, for improving the nonzero load balance of DSGD through a novel algorithm for balancing per-processor and per-epoch ratings. Their BaPabased DSGD shows a significant runtime improvement on small number of processors (< 16).…”

Section: Related Workmentioning

confidence: 99%

Scaling Stratified Stochastic Gradient Descent for Distributed Matrix Completion

Abubaker

Karsavuran

Aykanat

2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Stratified SGD (SSGD) is the primary approach for achieving serializable parallel SGD for matrix completion. State-of-the-art parallelizations of SSGD fail to scale due to large communication overhead. During an SGD epoch, these methods send data proportional to one of the dimensions of the rating matrix. We propose a framework for scalable SSGD through significantly reducing the communication overhead via exchanging point-to-point messages utilizing the sparsity of the rating matrix. We provide formulas to represent the essential communication for correctly performing parallel SSGD and we propose a dynamic programming algorithm for efficiently computing them to establish the point-to-point message schedules. This scheme, however, significantly increases the number of messages sent by a processor per epoch from O(K) to O(K 2 ) for a K-processor system which might limit the scalability. To remedy this, we propose a Hold-and-Combine strategy to limit the upper-bound on the number of messages sent per processor to O(K lgK). We also propose a hypergraph partitioning model that correctly encapsulates reducing the communication volume. Experimental results show that the framework successfully achieves a scalable distributed SSGD through significantly reducing the communication overhead. Our code is publicly available at: github.com/nfabubaker/CESSGD

show abstract

Section: Related Workmentioning

confidence: 99%

Scaling Stratified Stochastic Gradient Descent for Distributed Matrix Completion

Abubaker

Karsavuran

Aykanat

2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…In addition, distributed learning (DistL) also addresses those approaches that focus more on efficient use of computational resources in the presence of big data or large-scale modeling. Recently, some studies focus on developing efficient enablers for distributed learning e.g., matrix factorization techniques, distributed alternating direction method of multipliers [2], [3]. Some of those works shed light on privacy concerns [4].…”

Section: Introductionmentioning

confidence: 99%

A Survey and Guideline on Privacy Enhancing Technologies for Collaborative Machine Learning

et al. 2022

View full text Add to dashboard Cite

As machine learning and artificial intelligence (ML/AI) are becoming more popular and advanced, there is a wish to turn sensitive data into valuable information via ML/AI techniques revealing only data that is allowed by concerned parties or without revealing any information about the data to third parties. Collaborative ML approaches like federated learning (FL) help tackle these needs and concerns, bringing a way to use sensitive data without disclosing critically sensitive features of that data. In this paper, we provide a detailed analysis of state of the art for collaborative ML approaches from a privacy perspective. A detailed threat model and security and privacy considerations are given for each collaborative method. We deeply analyze Privacy Enhancing Technologies (PETs), covering secure multi-party computation (SMPC), homomorphic encryption (HE), differential privacy (DP), and confidential computing (CC) in the context of collaborative ML. We introduce a guideline on the selection of the privacy preserving technologies for collaborative ML and privacy practitioners. This study constitutes the first survey to provide an indepth focus on collaborative ML requirements and constraints for privacy solutions while also providing guidelines on the selection of PETs.

show abstract

Mapping the Landscape of Quantum Computing and High Performance Computing Research Over the Last Decade

Garcia-Buendia,

Muñoz-Montoro,

Cortina

et al. 2024

IEEE Access

View full text Add to dashboard Cite

BaPa: A Novel Approach of Improving Load Balance in Parallel Matrix Factorization for Recommender Systems

Cited by 6 publications

References 29 publications

Scaling Stratified Stochastic Gradient Descent for Distributed Matrix Completion

Scaling Stratified Stochastic Gradient Descent for Distributed Matrix Completion

A Survey and Guideline on Privacy Enhancing Technologies for Collaborative Machine Learning

Mapping the Landscape of Quantum Computing and High Performance Computing Research Over the Last Decade

Contact Info

Product

Resources

About