A Compress-Based Association Mining Algorithm for Large Dataset

Abstract

Abstract. The association mining is one of the primary sub-areas in the field of data mining. This technique had been used in numerous practical applications, including consumer market basket analysis, inferring patterns from web page access logs, network intrusion detection and pattern discovery in biological applications. Most of the traditional association-mining algorithms assume that whole dataset can be loaded in the main memory. Hence, problem arise when such algorithms is applied in large dataset. In this paper we present a new algorithm for association mining. Our algorithm is efficient when the size of dataset is huge that cannot be load in the main memory. The proposed algorithm also reduces the frequent itemsets search space, by eliminating non-frequent 1-itemsets after the first pass. Our performance evaluation shows algorithm outperforms Apriori algorithm in different datasets.

An Efficient Compression Algorithm for Uncertain Databases Aimed at Mining Problems

“…Compressing the transactions of databases is one way to solve the problem. For example, the work in [9] proposed a new approach to compress database to reduce the size of transactions' database. The algorithm was divided into data preprocessing and data mining.…”

“…Recently, many systems are proposed for knowledge discovery from compressed databases [9]- [11] that start by transform the original database into a new data representation where several transactions are merged to become a new transaction then it uses an Apriori-like algorithm for association rule mining to find useful information. However, there are some problems in the approach suggested by M. Ashrafi et al [9]; first, the compressed database is not reversible after the original database is transformed by the data preprocessing step.…”

“…However, there are some problems in the approach suggested by M. Ashrafi et al [9]; first, the compressed database is not reversible after the original database is transformed by the data preprocessing step. Second, although some rules can be mined from the new transactions, it still needs to scan the database again to verify the result.…”

BitTableFI: An efficient mining frequent itemsets algorithm

“…Ashrafi et al [3] presents an algorithm using a partition table structure to compress the database. The algorithm performs the support count on partition table not on database and reconstructs the partition table each iter-ation.…”

A Quantified Approach for large Dataset Compression in Association Mining

“…These techniques have been widely used to identify groups of genes sharing similar expression profiles and the results obtained so far have been extremely valuable. [3,5,7] However, the metrics adopted in these clustering techniques have discovered only a subset of relationships among the many potential relationship possible between the transcripts [9]. Clustering can work well when there is already a wealth of knowledge about the pathway in question, but it works less well when this knowledge is sparse [11].…”