Association rules based algorithm for identifying outlier transactions in data stream

Abstract

Most outlier detection algorithms are proposed to discover outlier patterns from static databases. Those algorithms are infeasible for instant identification of outlier patterns in data streams that continuously arriving and unbounded data serve as the data sources in many applications such as sensor data feeding. In this paper an association rules based method is proposed to find outlier patterns in data streams. The presented work segments transactions from data streams and then finds approximate frequent itemsets with single data scan instead of requiring multiple scans. Based on the derived association rules some transaction can be identified as outliers if their outlier degrees are higher than a predefined threshold. The proposed method not only just finds the outlier patterns but also identifies the most possible items that induce the abnormal transactions in the data streams. Efficiency comparisons with frequent itemsetsbased work are also done to verify the effectiveness of the proposed framework.

Unsupervised online detection and prediction of outliers in streams of sensor data

“…The high speeds of data flows in a stream requires an efficient processing of the stream [20,45,48,78]. If the observed values in a stream are analyzed, then the analysis has to be fast to provide the results in real time [1].…”

“…A typical definition of a stream is the following: it (1) contains a high volume of data [1,20,48], (2) does not have a distinctive termination, (3) is continuously updated [4] and (4) is unbounded [3,45,48]. The data cannot be stored entirely in the computer memory [48].…”

“…A stream can be utilized by using subsets of the stream. Outlier detection using the subsets is a local method [45] for outlier detection, because the data are used partially. The subsets of the data are assumed to contain the characteristics of the normal data.…”

Challenges and Issues in DATA Stream: A Review

“…An importing issue in data stream is instant recognition of outlier pattern. [26] Introduced an algorithm based on association rules mining. This algorithm utilizes prefixtree that regularly monitors the frequent items and traversal tree is used to find all the association rules.…”

“…Remove redundancy of nodes and speedup processing time. [26] How to find instant recognition of outlier pattern in data stream? association rules mining algorithm to solve the problem.…”