Mining frequent itemsets in data streams within a time horizon

Troiano, Luigi; Scibelli, Giacomo

doi:10.1016/j.datak.2013.10.002

Cited by 23 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(ii) The sliding window [10,13,14,33,37,52,57], which extracts the most recent transactions for analysis, the window usually has a fixed size, and the new arriving transaction results in that the oldest transaction is deleted from the window, where each transaction in the filled window has a constant weight. The representative algorithms include: Moment [15] designs an in-memory prefix-tree-based structure, called the Closed Enumeration Tree (CET), to maintain a dynamically selected set of itemsets over the sliding-window.…”

Section: Related Workmentioning

confidence: 99%

A Synopsis Based Approach for Itemset Frequency Estimation over Massive Multi-Transaction Stream

Wang

Cong

Zhang

et al. 2021

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

The streams where multiple transactions are associated with the same key are prevalent in practice, e.g., a customer has multiple shopping records arriving at different time. Itemset frequency estimation on such streams is very challenging since sampling based methods, such as the popularly used reservoir sampling, cannot be used. In this article, we propose a novel k -Minimum Value (KMV) synopsis based method to estimate the frequency of itemsets over multi-transaction streams. First, we extract the KMV synopses for each item from the stream. Then, we propose a novel estimator to estimate the frequency of an itemset over the KMV synopses. Comparing to the existing estimator, our method is not only more accurate and efficient to calculate but also follows the downward-closure property. These properties enable the incorporation of our new estimator with existing frequent itemset mining (FIM) algorithm (e.g., FP-Growth) to mine frequent itemsets over multi-transaction streams. To demonstrate this, we implement a KMV synopsis based FIM algorithm by integrating our estimator into existing FIM algorithms, and we prove it is capable of guaranteeing the accuracy of FIM with a bounded size of KMV synopsis. Experimental results on massive streams show our estimator can significantly improve on the accuracy for both estimating itemset frequency and FIM compared to the existing estimators.

show abstract

Section: Related Workmentioning

confidence: 99%

A Synopsis Based Approach for Itemset Frequency Estimation over Massive Multi-Transaction Stream

Wang

Cong

Zhang

et al. 2021

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

show abstract

“…Here is associate algorithm supported Apriori for knowledge stream mining referred to as Apriori with Window check and Window Itemset Shift(WIS) [7] that is given in algorithm 1, supported the window check, as advised by Propositions of windowing. Here in this case, the mining of frequent knowledge sets is additionally explained by both conditions-∃TI∈WS: I⊆TI and (ii) supp(I) ≥ S (see algorithm).…”

Section: Window Itemset Shift (Wis)mentioning

confidence: 99%

Shift Window FPTree - An Efficient Stream Mining Algorithm

Mishra¹,

Sharma²

2015

IJEME

View full text Add to dashboard Cite

Breathless flow in data collection and storage mechanism has enabled Firms to heap up a massive amount of data. In many cases, these huge volumes of data can be mined for fascinating and applicable information in a wide range of applications. When the arrival of data is fast as well in a large bunches in term of amount, this lead major problem to go through this data in both the circumstances in store it and in extracting the useful information from it. To taking under these issues continues mining or stream mining is a best way. Data steam mining allows to not storing the entire data for future prediction which lead to overcome the e-vestige and unnecessary storage overhead. But there is no such way in literature to mine continues data direct, so first one make it feasible accordingly and then mine it. Here in this paper we present an algorithm which handle stream data in very effective manner.

show abstract

“…However, to support the execution of more complex queries, there is the need to go beyond the domain of relational algebra. Indeed, complex applications such as frequent or rare pattern-mining on streams (useful for detecting security issues) [ 36 , 37 , 38 ] are not expressible in terms of relational algebra operators on tables (e.g., select, join), as well as operations involving the processing of packet payloads.…”

Section: Introductionmentioning

confidence: 99%

Data Stream Processing for Packet-Level Analytics

Fais

Lettieri

Procissi

et al. 2021

Sensors

View full text Add to dashboard Cite

One of the most challenging tasks for network operators is implementing accurate per-packet monitoring, looking for signs of performance degradation, security threats, and so on. Upon critical event detection, corrective actions must be taken to keep the network running smoothly. Implementing this mechanism requires the analysis of packet streams in a real-time (or close to) fashion. In a softwarized network context, Stream Processing Systems (SPSs) can be adopted for this purpose. Recent solutions based on traditional SPSs, such as Storm and Flink, can support the definition of general complex queries, but they show poor performance at scale. To handle input data rates in the order of gigabits per seconds, programmable switch platforms are typically used, although they offer limited expressiveness. With the proposed approach, we intend to offer high performance and expressive power in a unified framework by solely relying on SPSs for multicores. Captured packets are translated into a proper tuple format, and network monitoring queries are applied to tuple streams. Packet analysis tasks are expressed as streaming pipelines, running on general-purpose programmable network devices, and a second stage of elaboration can process aggregated statistics from different devices. Experiments carried out with an example monitoring application show that the system is able to handle realistic traffic at a 10 Gb/s speed. The same application scales almost up to 20 Gb/s speed thanks to the simple optimizations of the underlying framework. Hence, the approach proves to be viable and calls for the investigation of more extensive optimizations to support more complex elaborations and higher data rates.

show abstract

Mining frequent itemsets in data streams within a time horizon

Cited by 23 publications

References 8 publications

A Synopsis Based Approach for Itemset Frequency Estimation over Massive Multi-Transaction Stream

A Synopsis Based Approach for Itemset Frequency Estimation over Massive Multi-Transaction Stream

Shift Window FPTree - An Efficient Stream Mining Algorithm

Data Stream Processing for Packet-Level Analytics

Contact Info

Product

Resources

About