Parallelization with Multiplicative Algorithms for Big Data Mining

Luo, Dijun; Ding, Chris; Huang, Heng

doi:10.1109/icdm.2012.155

Cited by 17 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The interface e®ectively performs parallel execution of data mining algorithms such as k-means clustering, principal component analysis and linear regression based on Map-Reduce programming model. The problem of parallelising data mining and machine learning algorithms for handling big data sources has been tackled by Luo et al (2012) since the task of parallelisation is non-trivial. They proposed a strategy to parallelise series of data mining algorithms such as support vector machine and linear regression models using Map-Reduce programming models.…”

Section: Research Attempts Based On Local Pattern Analytics Strategymentioning

confidence: 99%

A Big Data Recommendation Engine Framework Based on Local Pattern Analytics Strategy for Mining Multi-Sourced Big Data

Venkatesan

Saravanan

Ramkumar

2019

J. Info. Know. Mgmt.

View full text Add to dashboard Cite

Organisations that perform business operations in a multi-sourced big data environment are in imperative need to discover meaningful patterns of interest from their diversified data sources. With the advent of big data technologies such as Hadoop and Spark, commodity hardwares play vital role in the task of data analytics and process the multi-sourced and multi-formatted big data in a reasonable cost and time. Though various data analytic techniques exist in the context of big data, recommendation system is more popular in web-based business applications to suggest suitable products, services, and items to potential customers. In this paper, we put forth a big data recommendation engine framework based on local pattern analytics strategy to explore user preferences and taste for both branch level and central level decisions. The framework encourages the practice of moving computing environment towards the data source location and avoids forceful integration of data. Further it assists decision makers to reap hidden preferences and taste of users from branch data sources for an effective customer campaign. The novelty of the framework has been evaluated in the benchmark dataset, MovieLens100k and results clearly confirm the advantages of the proposal.

show abstract

Section: Research Attempts Based On Local Pattern Analytics Strategymentioning

confidence: 99%

A Big Data Recommendation Engine Framework Based on Local Pattern Analytics Strategy for Mining Multi-Sourced Big Data

Venkatesan

Saravanan

Ramkumar

2019

J. Info. Know. Mgmt.

View full text Add to dashboard Cite

show abstract

“…MapReduce is a parallelizable data process framework aiming to provide a generic method to processing data on cluster or a grid. It has been used in many different areas such as graph pattern analysis [19,20], itemset mining [21], support vector machine [22] and also sequential pattern mining [23]. To make full use of computing resources and storage resources in cluster, HDFS is always used to store data files as multiple copies, so that MapReduce can take advantage of locality of data, and decrease data transmission time.…”

Section: Mapreducementioning

confidence: 99%

Towards efficient and scalable data mining using spark

Deng

Zhu

et al. 2014

2014 International Conference on Information and Communications Technologies (ICT 2014)

View full text Add to dashboard Cite

Following the requirements of discovery of valuable information from data increasing rapidly, data mining technologies have drawn people's attention for the last decade. However, the big data era makes even higher demands from the data mining technologies in terms of both processing speed and data amounts. Any data mining algorithm itself can hardly meet these requirements towards effective processing of big data, so distributed systems are proposed to be used. In this paper, a novel method of integrating a sequential pattern mining algorithm with a fast large-scale data processing engine Spark is proposed to mine patterns in big data. We use the well-known algorithm PrefixSpan as an example to demonstrate how this method helps handle massive data rapidly and conveniently. The experiments show that this method can make full use of cluster computing resources to accelerate the mining process, with a better performance than the common platform Hadoop.

show abstract

“…However, it is unable to "live in harmony" with the power grid", causing serious abandonment in the wind and solar energy [2], reducing the utilization efficiency of new energy based on wind and solar energy, which is seriously inconsistent the China source strategy of green, environmental protection and sustainable development [3]. In twenty-first Century, with the concept of big data and cloud computing put forward, the" energy revolution" of mankind has been greatly impacted [4][5][6].…”

Section: Introductionmentioning

confidence: 99%

Research on new energy utilization based on cloud computing and big data

Gao¹,

Zhang²,

Tong³

et al. 2018

Proceedings of the 2018 7th International Conference on Energy and Environmental Protection (ICEEP 2018)

View full text Add to dashboard Cite

Environmental, clean and sustainable new energy sources have been paid much attention by researchers all over the world due to the increasingly exhaustion of traditional energy resources and the worsening of the ecological environment. Wind and solar energy are typical new energy sources, they cannot "live in harmony" with the power grid because of wind and solar energy have great intermittent and volatility, which greatly reduced the utilization efficiency of new energy based on wind and solar energy. Initially, This paper analyzes the relationship between big data, cloud computing and new energy, and the development of large data, cloud computing and new energy; Then, parsing challenges of new energy big data; Last, through typical example in the new energy industry to realize the promoting effect of cloud computing and big data on new energy, proving the feasibility and efficiency of cloud computing and big data in the field of new energy.

show abstract

Parallelization with Multiplicative Algorithms for Big Data Mining

Cited by 17 publications

References 25 publications

A Big Data Recommendation Engine Framework Based on Local Pattern Analytics Strategy for Mining Multi-Sourced Big Data

A Big Data Recommendation Engine Framework Based on Local Pattern Analytics Strategy for Mining Multi-Sourced Big Data

Towards efficient and scalable data mining using spark

Research on new energy utilization based on cloud computing and big data

Contact Info

Product

Resources

About