Mining Frequent Itemsets in a Stream

Abstract

Mining frequent itemsets in a datastream proves to be a difficult problem, as itemsets arrive in rapid succession and storing parts of the stream is typically impossible. Nonetheless, it has many useful applications; e.g., opinion and sentiment analysis from social networks. Current stream mining algorithms are based on approximations. In earlier work, mining frequent items in a stream under the max-frequency measure proved to be effective for items. In this paper, we extended our work from items to itemsets. Firstly, an optimized incremental algorithm for mining frequent itemsets in a stream is presented. The algorithm maintains a very compact summary of the stream for selected itemsets. Secondly, we show that further compacting the summary is nontrivial. Thirdly, we establish a connection between the size of a summary and results from number theory. Fourthly, we report results of extensive experimentation, both of synthetic and real-world datasets, showing the efficiency of the algorithm both in terms of time and space.

MARBLES: Mining Association Rules Buried in Long Event Sequences

“…Further interestingness measures for episodes, either statistically motivated or aimed at removing bias towards smaller episodes, were made by Garriga [5], Gwadera et al [10,11], Calders et al [4], and Tatti [17]. All these methods, however, were limited to finding interesting episodes, and stopped short of discovering association rules between them.…”

“…First of all, the confidence of rule G ⇒ H would depend on our choice of disjoint minimal windows -if we chose the first minimal window of H, s [1,5], we would find two occurrences of G outside it and the confidence of the rule would be 1 3 , whereas if we chose the second minimal window of H, s [4,10], we would find just one occurrence of G outside it, and the confidence would be 2 3 . More importantly, whichever choice we made, we would not be able to get the correct result, showing that every occurrence of G is contained within an occurrence of H. Now that we have seen that we cannot define the confidence of an association rule using either the disjoint-window frequencies, or the containment of the disjoint occurrences, of the two episodes, we are ready to present a definition that corresponds exactly to our intuition.…”

“…Consider sequence s = axybca and episodes X = b → c and Y = {a, b → c}. Sequence s contains one minimal window of X, namely s [4,5]. This occurrence of X can be extended in search of an occurrence of Y .…”

“…This occurrence of X can be extended in search of an occurrence of Y . There are two candidate minimal windows of Y that can be considered, s [1,5] and s [4,6]. If we choose the former, the extensibility of s [4,5] into Y would equal 2 5 , and if we use the latter, this value would rise up to 2 3 .…”

“…There are two candidate minimal windows of Y that can be considered, s [1,5] and s [4,6]. If we choose the former, the extensibility of s [4,5] into Y would equal 2 5 , and if we use the latter, this value would rise up to 2 3 . Since we are interested in how far we need to look in order to extend an occurrence of X into an occurrence of Y , we clearly need to look for the smallest minimal window of Y that satisfies the conditions.…”