Distributed Frequent Itemset Mining Using Size Based Assignment Technique

Abstract

Distributed data mining is attracting researchers due to the globalisation and increase in the distributed databases. Very little work has been done in this area. Algorithms which are available, mostly first partition the database, distribute them amongst different sites for parallel processing. In the real life scenario data generated at different sites are not under control of centralised database and the numbers of transactions at each site are highly varied. Due to this some sites are heavily loaded and some sites are comparatively free. A novel approach, size based assignment, is proposed in this paper which takes care of the database size available at each site while distributing the load for finding the global frequent itemsets. It also reduces the communication load by pruning and no-broadcasting techniques. The algorithm is compared with similar algorithms on execution time. Results show that the new technique performed best amongst them in time execution.