Scalable Collaborative Filtering Recommendation Algorithm with MapReduce

Shang, Yang; Li, Zhiyang; Qu, Wenyu; Xu, Yujie; Song, Zining; Zhou, Xuefei

doi:10.1109/dasc.2014.27

Cited by 8 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then the difference between the average and the mean values is calculated. Shown in equation (6). ̅ (6) 3.…”

Section: Collaborative Filteringmentioning

confidence: 99%

“…In the scale-out approach, an additional computer node is used to run a recommendation system to obtain good scalability. The scale-out method implemented in previous research was using MapReduce Hadoop as practiced by [4], [5], [6], and [7] to get good scalability from traditional collaborative filtering recommendation systems. Another study was conducted by [8] who used Apache Spark to implement a scale-out approach to overcome the scalability of the recommendation system with traditional collaborative filtering methods.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parallelization of Hybrid Content Based and Collaborative Filtering Method in Recommendation System with Apache Spark

Ikhsanudin

Winarko²

2019

Indonesian J. Comput. Cybern. Syst.

View full text Add to dashboard Cite

Collaborative Filtering as a popular method that used for recommendation system. Improvisation is done in purpose of improving the accuracy of the recommendation. A way to do this is to combine with content based method. But the hybrid method has a lack in terms of scalability. The main aim of this research is to solve problem that faced by recommendation system with hybrid collaborative filtering and content based method by applying parallelization on the Apache Spark platform.Based on the test results, the value of hybrid collaborative filtering method and content based on Apache Spark cluster with 2 node worker is 1,003 which then increased to 2,913 on cluster having 4 node worker. The speedup got more increased to 5,85 on the cluster that containing 7 node worker.

show abstract

“…Then the difference between the average and the mean values is calculated. Shown in equation (6). ̅ (6) 3.…”

Section: Collaborative Filteringmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Parallelization of Hybrid Content Based and Collaborative Filtering Method in Recommendation System with Apache Spark

Ikhsanudin

Winarko²

2019

Indonesian J. Comput. Cybern. Syst.

View full text Add to dashboard Cite

show abstract

“…Interim, a soft assignment tool for the hierarchical inverted index has been discussed resulting in fixation of decrease in accuracy of recommendation caused by the index. The MapReduce are implemented in both real data and simulated data, proving the implementation has the potential to scale to vast number of items and users which has been assured the recommendation accuracy [7,23].…”

Section: Related Workmentioning

confidence: 99%

An Improved Hybrid Distributed Collaborative Filtering Model for Recommender Engine using Apache Spark

Lenka¹,

Barik²,

Panigrahi³

et al. 2018

IJISA

View full text Add to dashboard Cite

The present scenario there is a serious need of scalability for efficient analytics of big data. In order to achieve this, technology like MapReduce, Pig and HIVE came into action but when the question comes to scalability; Apache Spark maintains a great position far ahead. In this research paper, it has designed and developed an improved hybrid distributed collaborative model for filtering recommender engine. Execution time, scalability and robustness of the engine are the three evaluation parameters; has been considered for this present study. The present work keeps an eye on recommender system built with help of Apache Spark. Apart from this, it has been proposed and implemented the bisecting KMeans clustering algorithms. It has discussed about the comparative analysis between KMeans and Bisecting KMeans clustering algorithms on Apache Spark environment.

show abstract

“…Now, the MR programming model has been successfully applied in many fields such as web data mining, large scale documents analytics, query processing, bioinformatics, financial prediction, social network analytics, recommendation algorithm , clustering algorithm , privacy‐preserving algorithm , and so on.…”

Section: Background and Related Workmentioning

confidence: 99%

HybridMR: a new approach for hybrid MapReduce combining desktop grid and cloud infrastructures

Tang

Fedak

2015

Concurrency and Computation

View full text Add to dashboard Cite

Summary This paper introduces HybridMR, a novel model for the execution of MapReduce (MR) computation on hybrid computing environment. Using this model, high performance cloud resources and heterogeneous desktop personal computers (PCs) in Internet or Intranet can be integrated to form a hybrid computing environment. Thanks to HybridMR, the computation and storage capability of large scale desktop PCs can be fully utilized to process large scale datasets. HybridMR relies on two innovative solutions to enable such large scale data‐intensive computation. The first one is HybridDFS, which is a hybrid distributed file system. HybridDFS features reliable distributed storage that alleviates the volatility of desktop PCs, thanks to fault tolerance and file replication mechanism. The second innovation is a new node priority‐based fair scheduling (NPBFS) algorithm has been developed in HybridMR to achieve both data storage balance and job assignment balance by assigning each node a priority through quantifying CPU speed, memory size, and input and output capacity. In this paper, we describe the HybridMR, HybridDFS, and NPBFS. We report on performance evaluation results, which show that the proposed HybridMR not only achieves reliable MR computation, reduces task response time, and improves the performance of MR, but also reduces the computation cost and achieves a greener computing mode. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

Scalable Collaborative Filtering Recommendation Algorithm with MapReduce

Cited by 8 publications

References 16 publications

Parallelization of Hybrid Content Based and Collaborative Filtering Method in Recommendation System with Apache Spark

Parallelization of Hybrid Content Based and Collaborative Filtering Method in Recommendation System with Apache Spark

An Improved Hybrid Distributed Collaborative Filtering Model for Recommender Engine using Apache Spark

HybridMR: a new approach for hybrid MapReduce combining desktop grid and cloud infrastructures

Contact Info

Product

Resources

About