FP-outlier: Frequent pattern based outlier detection

Abstract

An outlier in a dataset is an observation or a point that is considerably dissimilar to or inconsistent with the remainder of the data. Detection of such outliers is important for many applications and has recently attracted much attention in the data mining research community. In this paper, we present a new method to detect outliers by discovering frequent patterns (or frequent itemsets) from the data set. The outliers are defined as the data transactions that contain less frequent patterns in their itemsets. We define a measure called FPOF (Frequent Pattern Outlier Factor) to detect the outlier transactions and propose the FindFPOF algorithm to discover outliers. The experimental results have shown that our approach outperformed the existing methods on identifying interesting outliers.

ZERO++: Harnessing the Power of Zero Appearances to Detect Anomalies in Large-Scale Data Sets

“…Most existing categorical data oriented methods are based on a general assumption that anomalies lie in regions of low frequency (Akoglu et al, 2012;Ghoting, Otey, & Parthasarathy, 2004;He et al, 2005;Koufakou, Ortiz, Georgiopoulos, Anagnostopoulos, & Reynolds, 2007;Koufakou & Georgiopoulos, 2010;Smets & Vreeken, 2011;He, Deng, Xu, & Huang, 2006). Typical examples are frequent patterns based methods FPOF (He et al, 2005) and infrequent patterns based methods LOADED (Ghoting et al, 2004). FPOF and LOADED build a single model on the entire training set, and identify anomalies based on frequent patterns and infrequent patterns, respectively.…”

“…We compared ZERO++ with FPOF (He et al, 2005), COMPREX (Akoglu et al, 2012), iForest (Liu et al, 2012) and LOF (Breunig et al, 2000). FPOF is a state-of-the-art frequency-based method for categorical data.…”

“…ZERO++ is unique in that it works in regions of subspaces that are not occupied by data; whereas existing methods (Akoglu, Tong, Vreeken, & Faloutsos, 2012;Breunig, Kriegel, Ng, & Sander, 2000;He, Xu, Huang, & Deng, 2005;Liu, Ting, & Zhou, 2012) identify anomalies based on the assumption that anomalies lie in regions of low density/frequency, i.e., in regions occupied by data.…”

“…Frequency-based algorithms (Akoglu et al, 2012;He et al, 2005;Smets & Vreeken, 2011) need to conduct a subspace pattern searching which have time and space complexities that are at least quadratic in terms of the data dimensionality. ZERO++ involves no searching; thus it runs significantly faster.…”

Mining Contrast Subspaces

“…There are a few density-based outlier detection methods, such as [9][10][11][12]. Our method is inherently different from those, since we do not target at outlier objects at all.…”