A general-purpose distributed pattern mining system

Belhadi, Asma; Djenouri, Youcef; Lin, Jerry Chun‐Wei; Cano, Alberto

doi:10.1007/s10489-020-01664-w

Cited by 19 publications

(15 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this is not a realistic scenario, and the objective is to decrease the number of items shared by the separated groups. Existing work [5] identified that k-means [41] and DBSCAN [42] can obtain a good performance of transaction decomposition, and k -means showed better results than that of the DBSCAN. Thus, a k-means model is used in the designed framework for transaction decomposition that can group highly relevant transactions in the same group.…”

Section: Propositionmentioning

confidence: 99%

“…As there is rapid growth of information technologies regarding machine learning models, Internet of Things (IoT) [1], and edge and cloud computing [2,3], data-driven mining has become an important topic that can be used to extract the meaningful information from the collections of those techniques. Several pattern mining models [4][5][6][7][8][9] have been extensively studied, and the most fundamental knowledge of pattern mining in knowledge discovery in databases (KDD) is called ARM or association rule mining, which is deployed through varied applications and specific domains. Among them, Apriori was presented for finding the association rules set in transactional databases iteratively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mining Profitable and Concise Patterns in Large-Scale Internet of Things Environments

Lin

Djenouri

Srivastava

et al. 2021

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

In recent years, HUIM (or a.k.a. high-utility itemset mining) can be seen as investigated in an extensive manner and studied in many applications especially in basket-market analysis and its relevant applications. Since current basket-market scenario also involves IoT equipment to collect information, i.e., sensor or smart devices, it is necessary to consider the mining of HUIs (or a.k.a. high-utility itemsets) in a large-scale database especially with IoT situations. First, a GA-based MapReduce model is presented in this work known as GMR-Miner for mining closed patterns with high utilization in large-scale databases. The k -means model is initially adopted to group transactions regarding their relevant correlation based on the frequency factor. A genetic algorithm (GA) is utilized in the developed MapReduce framework that can be used to explore the potential and possible candidates in a limited time. Also, the developed 3-tier MapReduce model can be easily deployed in Spark for the handlings of any database of large scale for knowledge discovery of closed patterns with high utilization. We created sets of extensive experimental environments for evaluating the results of the developed GMR-Miner compared to the well-known and state-of-the-art CLS-Miner. We present our in-depth results to show that the developed GMR-Miner outperforms CLS-Miner in many criteria, i.e., memory usage, scalability, and runtime.

show abstract

Section: Propositionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Mining Profitable and Concise Patterns in Large-Scale Internet of Things Environments

Lin

Djenouri

Srivastava

et al. 2021

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…It integrates "Density-Based Spatial Clustering of Applications and distributed computing represented, CPU multi-cores and Single CPU for solving pattern mining problems". Performance of any algorithm [14]is also effected by the number of nodes in the distributed data system. With the increase of the transactions or nodes, the performance improves.…”

Section: Related Workmentioning

confidence: 99%

“…In the distributed setup,data is generated or createdat different locations. The number of transactions at various sites differ [14] from a few hundred to millions of transactions.In this setup, resources at each site is also limited and are distributed. Data mining requires great amounts of resources [27] so technique for flexible distribution of work load amongst sites needs to be developed.…”

Section: Sizebased Assignment (Sba) Technique For Polling Site Assignmentmentioning

confidence: 99%

Distributed Frequent Itemset Mining Using Size Based Assignment Technique

2020

IJETER

View full text Add to dashboard Cite

Distributed data mining is attracting researchers due to the globalisation and increase in the distributed databases. Very little work has been done in this area. Algorithms which are available, mostly first partition the database, distribute them amongst different sites for parallel processing. In the real life scenario data generated at different sites are not under control of centralised database and the numbers of transactions at each site are highly varied. Due to this some sites are heavily loaded and some sites are comparatively free. A novel approach, size based assignment, is proposed in this paper which takes care of the database size available at each site while distributing the load for finding the global frequent itemsets. It also reduces the communication load by pruning and no-broadcasting techniques. The algorithm is compared with similar algorithms on execution time. Results show that the new technique performed best amongst them in time execution.

show abstract

“…Although these examples are perfectly capable of fulfilling the sequential pattern mining task, traditional algorithms suffer greatly with runtime and accuracy when dealing with massive datasets [11]. Another drawback of the frequent pattern mining solutions is that their output data are proved to be challenging to interpret and handle-especially when the number of the mined sequences is high-often introducing a new problem to solve [12].…”

Section: Introductionmentioning

confidence: 99%

Event-Tree Based Sequence Mining Using LSTM Deep-Learning Model

2021

View full text Add to dashboard Cite

During the operation of modern technical systems, the use of the LSTM model for the prediction of process variable values and system states is commonly widespread. The goal of this paper is to expand the application of the LSTM-based models upon obtaining information based on prediction. In this method, by predicting transition probabilities, the output layer is interpreted as a probability model by creating a prediction tree of sequences instead of just a single sequence. By further analyzing the prediction tree, we can take risk considerations into account, extract more complex prediction, and analyze what event trees are yielded from different input sequences, that is, with a given state or input sequence, the upcoming events and the probability of their occurrence are considered. In the case of online application, by utilizing a series of input events and the probability trees, it is possible to predetermine subsequent event sequences. The applicability and performance of the approach are demonstrated via a dataset in which the occurrence of events is predetermined, and further datasets are generated with a higher-order decision tree-based model. The case studies simply and effectively validate the performance of the created tool as the structure of the generated tree, and the determined probabilities reflect the original dataset.

show abstract

A general-purpose distributed pattern mining system

Cited by 19 publications

References 75 publications

Mining Profitable and Concise Patterns in Large-Scale Internet of Things Environments

Mining Profitable and Concise Patterns in Large-Scale Internet of Things Environments

Distributed Frequent Itemset Mining Using Size Based Assignment Technique

Event-Tree Based Sequence Mining Using LSTM Deep-Learning Model

Contact Info

Product

Resources

About