Efficient mining of association rules in distributed databases

Abstract

Many sequential algorithms have been proposed for mining of association rules. However, very little work has been done in mining association rules in distributed databases. A direct application of sequential algorithms to distributed databases is not effective, because it requires a large amount of communication overhead. In this study, an efficient algorithm, DMA, is proposed. It generates a small number of candidate sets and requires only O(n) messages for support count exchange for each candidate set, where n is the number of sites in a distributed database. The algorithm has been implemented on an experimental test bed and its performance is studied. The results show that DMA has superior performance when comparing with the direct application of a popular sequential algorithm in distributed databases.

Efficient Classification from Multiple Heterogeneous Databases

“…Either way of distribution divides the rows or columns of a table into different parts. Distributed data mining approaches for horizontally partitioned data include meta-learning [3] that merges models built from different sites, and privacy preserving techniques including decision tree [8] and association rule mining [7]. Those for vertically partitioned data include association rule mining [12] and k-means clustering [13].…”

“…However, perfect integration of heterogeneous data sources is a very challenging problem, and it is often impossible to migrate one whole database to another site. In contrast, distributed data mining [3,7,8,12,13] aims at discovering knowledge from a dataset that is stored at different sites. But they focus on a homogeneous dataset (a single table or a set of transactions) that is distributed to multiple sites, thus are unable to handle heterogeneous relational databases.…”

Distributed Frequent Closed Itemsets Mining

“…[14][15][16][17][18][19] have focused on parallel and distributed algorithms for mining association rules, but not the closed itemsets mining. In [1], an algorithm of synthesizing high frequency rules from different data sources based on weight model is proposed.…”

A Compress-Based Association Mining Algorithm for Large Dataset

“…One of the most important fields of the datamining domain is the association mining. Many interesting and efficient mining of association rule algorithms have been proposed in the different mining literature [2,3,4,5,6,8,9,10,11]. In this paper we present an efficient association-mining algorithm for large dataset.…”