Methods for finding frequent items in data streams

Cormode, Graham; Hadjieleftheriou, Marios

doi:10.1007/s00778-009-0172-z

Cited by 144 publications

(105 citation statements)

References 27 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…The task is to find all items whose frequency exceeds a specified threshold in the data stream [14]. Significant effort has been dedicated to this research topic in the past few decades, and most of the research community seems to have converged on a counter-based approach [34] [39] [41].…”

Section: Related Workmentioning

confidence: 99%

“…Although the counter-based methods have good approximation in finding the frequent items, they are designed for discrete items only, not for mining the real-value items which our system is aiming at. The literature has offered some other methods for frequent items, such as the sketch-based methods which use bit-maps of counters to estimate the frequency for each item in the data [12][20] [31], and quantile-based algorithms which find the frequent items through the quantile algorithm [14] [56]. However, all the algorithms assume the items are discrete and countable, and thus are not readily applicable to our system.…”

Section: Related Workmentioning

confidence: 99%

“…Note that the problem of maintaining discrete frequent items from unbounded streams in bounded space is known to be unsolvable in general, and thus has opened up an active area of research in approximation algorithms for this task [14]. However,…”

Section: Frequent Pattern Maintenancementioning

confidence: 99%

“… Tractability: The high dimensionality of the data objects, combined with the inability to avail of common techniques and representations for discrete frequent pattern mining (hashing, graphs, trees, and lattices [14]) seems to bode ill for our hopes to produce a highly tractable algorithm.…”

Section: Frequent Pattern Maintenancementioning

confidence: 99%

“…Frequent item mining algorithms for discrete data must handle million-plus Hertz arrival rates [14]. We begin with the intuition of our solution to these problems.…”

Section: Frequent Pattern Maintenancementioning

confidence: 99%

See 4 more Smart Citations

A general framework for never-ending learning from time series streams

Chen

Hao

Rakthanmanon

et al. 2014

Data Min Knowl Disc

View full text Add to dashboard Cite

Abstract-Time series classification has been an active area of research in the data mining community for over a decade, and significant progress has been made in the tractability and accuracy of learning. However, virtually all work assumes a one-time training session in which labeled examples of all the concepts to be learned are provided. This assumption may be valid in a handful of situations, but it does not hold in most medical and scientific applications where we initially may have only the vaguest understanding of what concepts can be learned. Based on this observation, we propose a never-ending learning framework for time series in which an agent examines an unbounded stream of data and occasionally asks a teacher (which may be a human or an algorithm) for a label. We demonstrate the utility of our ideas with experiments that consider real-world problems in domains as diverse as medicine, entomology, wildlife monitoring, and human behavior analyses.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Frequent Pattern Maintenancementioning

confidence: 99%

Section: Frequent Pattern Maintenancementioning

confidence: 99%

“…Frequent item mining algorithms for discrete data must handle million-plus Hertz arrival rates [14]. We begin with the intuition of our solution to these problems.…”

Section: Frequent Pattern Maintenancementioning

confidence: 99%

See 3 more Smart Citations

A general framework for never-ending learning from time series streams

Chen

Hao

Rakthanmanon

et al. 2014

Data Min Knowl Disc

View full text Add to dashboard Cite

show abstract

Technologies for Dealing with Information Overload: An Engineer's Point of View

Calders

Fletcher

Kamiran

et al. 2012

Information Overload

View full text Add to dashboard Cite

A novel sensitive DDoS attacks against statistical test in network traffic fusion

Kulandaivel,

Kumar,

Sathiyamoorthi

et al. 2023

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Distributed denial of service (DDoS) attacks are common attacks that limit casual users from cracking critical network resources. A method to find the DDoS attack comes to light as an essential and hot research issue. Therefore, the increment in network usage scale demands more challenges to see the DDoS attack. Non‐real‐time collection of network traffic will make sincere complications inaccuracy; DDoS attack detection, and efficiency. Lately, sketch data structures play an excellent role in high‐speed networks to suppress and combine network traffic issues. However, the sketch data structure has issues rebuilding the reverse keys that behave abnormally due to the reversible hash function. To rectify the above difficulties, we include the Chinese Remainder Theorem‐based Reverse Sketching (CRTRS). However, Reverse Sketching is not good at suppressing and combining heavy network traffic, but it can easily detect abnormal keys. According to the network traffic records setup by CRTRS, we add the Modified Cumulative Sum Quality Control Scheme (MCSQCS) that encourages DDoS attack detection. The enforcement of the proposed work with DDoS attack detection has two kind of different data sets. The result of the experiments shows that it gives more accuracy and efficiency in detecting DDoS attacks. Therefore, comparing other DDoS detecting methods with the sketch data structure has fewer computational complications. Improving the anomalous keys is the most critical challenge in this proposed work. To illustrate the accuracy rate, we experiment with a new technique called CIC‐Friday Heavy network traffic fragment. We also explain the experimental results of the MAWI heavy network fragment with the various parameter setups. To compare our proposed method with other similar existing methods, we use a sketch data structure to manage the network traffic data. It also uses the MSPCA method to detect the DDoS attack. The information‐based analysis helps to see the DDoS attack. The result analysis of True Positive Value (TPV) and False Positive Value (FPV) with MSPCA techniques has 79% and .1% accuracy. Similarly, information‐based detection has an accuracy of about 90% and 5%. Hence, the proposed method gives more accuracy for the detection of DDoS attacks.

show abstract

Methods for finding frequent items in data streams

Cited by 144 publications

References 27 publications

A general framework for never-ending learning from time series streams

A general framework for never-ending learning from time series streams

Technologies for Dealing with Information Overload: An Engineer's Point of View

A novel sensitive DDoS attacks against statistical test in network traffic fusion

Contact Info

Product

Resources

About