Parallel CCD++ on GPU for Matrix Factorization

Nisa, Israt; Sukumaran-Rajam, Aravind; Kunchum, Rakshith; Sadayappan, P.

doi:10.1145/3038228.3038240

Cited by 21 publications

(9 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The optimization problem for MF is a classic nonconvex problem [8,17]. An alternative minimization strategy, e.g., Alternating Least Squares (ALS) [54], SGD [59] [64], or Cyclic Coordinate Descent (CCD) [47], is adopted to solve this nonconvex problem. An efficient big data processing method requires highly efficient hardware and algorithms.…”

Section: Rrlated Workmentioning

confidence: 99%

Locality Sensitive Hash Aggregated Nonlinear Neighbourhood Matrix Factorization for Online Sparse Big Data Analysis

Li¹,

Li²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Matrix factorization (MF) can extract the low-rank features and integrate the information of the data manifold distribution from high-dimensional data, which can consider the nonlinear neighbourhood information. Thus, MF has drawn wide attention for low-rank analysis of sparse big data, e.g., Collaborative Filtering (CF) Recommender Systems, Social Networks, and Quality of Service. However, the following two problems exist: 1) huge computational overhead for the construction of the Graph Similarity Matrix (GSM), and 2) huge memory overhead for the intermediate GSM. Therefore, GSM-based MF, e.g., kernel MF, graph regularized MF, etc., cannot be directly applied to the low-rank analysis of sparse big data on cloud and edge platforms. To solve this intractable problem for sparse big data analysis, we propose Locality Sensitive Hashing (LSH) aggregated MF (LSH-MF), which can solve the following problems: 1)The proposed probabilistic projection strategy of LSH-MF can avoid the construction of the GSM. Furthermore, LSH-MF can satisfy the requirement for the accurate projection of sparse big data. 2) To run LSH-MF for fine-grained parallelization and online learning on GPUs, we also propose CULSH-MF, which works on CUDA parallelization. Experimental results show that CULSH-MF can not only reduce the computational time and memory overhead but also obtain higher accuracy. Compared with deep learning models, CULSH-MF can not only save training time but also achieve the same accuracy performance.

show abstract

Section: Rrlated Workmentioning

confidence: 99%

Locality Sensitive Hash Aggregated Nonlinear Neighbourhood Matrix Factorization for Online Sparse Big Data Analysis

Li¹,

Li²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The algorithm has two versions of parallelization on different machines: one version for multi-core shared memory systems and the other for distributed systems [2]. Recently Nisa et al improve the CCD++ method on GPUs with loop fusion and tiling [45]. Yang et al present an efficient and portable CDMF solver on modern multi-core and many-cores [46].…”

Section: Related Workmentioning

confidence: 99%

clMF: A fine-grained and portable alternating least squares algorithm for parallel matrix factorization

Chen

Fang

Liu

et al. 2020

Future Generation Computer Systems

View full text Add to dashboard Cite

Alternating least squares (ALS) has been proved to be an effective solver for matrix factorization in recommender systems. To speed up factorizing performance, various parallel ALS solvers have been proposed to leverage modern multi-cores and many-cores. Existing implementations are limited in either speed or portability. In this paper, we present an efficient and portable ALS solver (clMF) for recommender systems. On one hand, we diagnose the baseline implementation and observe that it lacks of the awareness of the hierarchical thread organization on modern hardware. To achieve high performance, we apply the thread batching technique, the fine-grained tiling technique and three architecture-specific optimizations. On the other hand, we implement the ALS solver in OpenCL so that it can run on various platforms (CPUs, GPUs and MICs). Based on the architectural specifics, we select a suitable code variant for each platform to efficiently map it to the underlying hardware. The experimental results show that our implementation performs 2.8×-15.7× faster on an Intel 16-core CPU, 23.9×-87.9× faster on an NVIDIA K20C GPU and 34.6×-97.1× faster on an AMD Fury X GPU than the baseline implementation. On the K20C GPU, our implementation also outperforms cuMF over different latent features ranging from 10 to 100 with various real-world recommendation datasets.

show abstract

“…A significant amount of research has been conducted on sparse tensor operations in terms of storage, computation, and performance in this decade. Tensor factorization is gaining significant popularity, like matrix factorization [30], [31], [32]. In this section, we briefly discuss prior work on MTTKRP Fig.…”

Section: Related Workmentioning

confidence: 99%

Load-Balanced Sparse MTTKRP on GPUs

Nisa

Sukumaran-Rajam

et al. 2019

2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

Self Cite

View full text Add to dashboard Cite

Sparse matricized tensor times Khatri-Rao product (MTTKRP) is one of the most computationally expensive kernels in sparse tensor computations. This work focuses on optimizing the MTTKRP operation on GPUs, addressing both performance and storage requirements. We begin by identifying the performance bottlenecks in directly extending the state-ofthe-art CSF (compressed sparse fiber) format from CPUs to GPUs. A significant challenge with GPUs compared to multicore CPUs is that of utilizing the much greater degree of parallelism in a load-balanced fashion for irregular computations like sparse MTTKRP. To address this issue, we develop a new storage-efficient representation for tensors that enables highperformance, load-balanced execution of MTTKRP on GPUs. A GPU implementation of sparse MTTKRP using the new sparse tensor representation is shown to outperform all currently known parallel sparse CPU and GPU MTTKRP implementations.

show abstract

Parallel CCD++ on GPU for Matrix Factorization

Cited by 21 publications

References 24 publications

Locality Sensitive Hash Aggregated Nonlinear Neighbourhood Matrix Factorization for Online Sparse Big Data Analysis

Locality Sensitive Hash Aggregated Nonlinear Neighbourhood Matrix Factorization for Online Sparse Big Data Analysis

clMF: A fine-grained and portable alternating least squares algorithm for parallel matrix factorization

Load-Balanced Sparse MTTKRP on GPUs

Contact Info

Product

Resources

About