Verifying and Mining Frequent Patterns from Large Windows over Data Streams

Abstract

Mining frequent itemsets from data streams has proved to be very difficult because of computational complexity and the need for real-time response. In this paper, we introduce a novel verification algorithm which we then use to improve the performance of monitoring and mining tasks for association rules. Thus, we propose a frequent itemset mining method for sliding windows, which is faster than the state-of-the-art methods-in fact, its running time that is nearly constant with respect to the window size entails the mining of much larger windows than it was possible before. The performance of other frequent itemset mining methods (including those on static data) can be improved likewise, by replacing their counting methods (e.g., those using hash trees) by our verification algorithm.

Frequent itemset mining over stream data: overview

“…In 2008, Mozafari developed one AOG-based algorithm [28]: Lightweight frequent pattern mining named as LWF. It enables to find most of frequent itemsets with an approximate solution over the incoming stream data by using adaptation and releasing the least frequent itemsets regularly in order to count more frequent itemsets.…”

“…Most of the achievements related to frequent itemset mining in stream data [21][22][23][24][25][26][27][28][29][30][31] focus on this issue. In 2002, Datar proposed Ref.…”

A Sliding Window Algorithm for Relational Frequent Patterns Mining from Data Streams

“…The proposed approach is based on the sliding window model, which completely discard stale data, thus saving memory storage and facilitating the detection of the distribution drift. This model is common to several algorithms for frequent pattern mining in data streams [9,11,6,13]. However, all these algorithms work on a single database relation (propositional representation) and are not able to deal directly with complex data stored in multiple database relations.…”

Continuous Post-Mining of Association Rules in a Data Stream Management System

“…In fact, the main reason that most of the above algorithms are not suitable for data streams is because they require several passes through the data, or they are computationally too expensive. Thus, new algorithms have been proposed for mining of frequent patterns in the context of data streams [8,15,18,28,9,24]. Due to space limitations, here we will only discuss those that are most relevant to this chapter.…”

“…This efficient algorithm, however, leaves much to be desired for mining data streams; in fact, it requires two passes over each window (one for finding the frequent items and another for finding the frequent itemsets), and becomes prohibitively expensive for large windows. Our proposed verifiers [24] borrow this fp-tree structure and the conditionalization idea from [12].…”