Fast and memory efficient mining of frequent closed itemsets

Abstract

Abstract-This paper presents a new scalable algorithm for discovering closed frequent itemsets, a lossless and condensed representation of all the frequent itemsets that can be mined from a transactional database.Our algorithm exploits a divide-and-conquer approach and a bitwise vertical representation of the database, and adopts a particular visit and partitioning strategy of the search space based on an original theoretical framework, which formalizes the problem of closed itemsets mining in detail. The algorithm adopts several optimizations aimed to save both space and time in computing itemset closures and their supports. In particular, since one of the main problems raising up in this type of algorithms is the multiple generation of the same closed itemset, we propose a new effective and memory-efficient pruning technique, which, unlike other previous proposals, does not require the whole set of closed patterns mined so far to be kept in the main memory. This technique also permits each visited partition of the search space to be mined independently in any order and thus also in parallel.The tests conducted on many publicly available datasets show that our algorithm is scalable and outperforms other state-ofthe-art algorithms like CLOSET+ and FP-CLOSE, in some cases by more than one order of magnitude. More importantly, the performance improvements become more and more significant as the support threshold is decreased.

Updating mined class association rules for record insertion

“…7,8,9,10,11,12,13,14,15,16,17,18,19,to 20 show that CAR-Incre is more efficient than CAR-Miner in most cases, especially in large datasets or large minSup. Examples are Poker-hand (a large number of records) or Chess (minSup is large).…”

“…Pattern mining has widely used applications in a lot of areas such as association rule mining [4,13,18,24], sequence mining [19,21], and others [3,14]. Association rule mining is to mine relationships among items in a transaction database.…”

Mining Top-K Association Rules

“…The first optimization is to use bit vectors for representing tidsets and database transactions (when the database fits into memory). The benefits of using bit vectors is that it can greatly reduce the memory used and that the intersection of two tidsets can be done very efficiently with the logical AND operation [13]. The second optimization is to implement L and R with data structures supporting efficient insertion, deletion and finding the smallest element and maximum element.…”

Analysis of Dynamic Sensor Networks: Power Law Then What?

“…The functions of the Galois connection are thus used to compute the closure of the enumerated set and the associated component on the other dimension. Several techniques have been proposed to guarantee that each formal concept is uniquely generated [10].…”