An Outlier Detection Method Based on Clustering

Abstract

Abstract-In this paper we propose a clustering based method to capture outliers. We apply K-means clustering algorithm to divide the data set into clusters. The points which are lying near the centroid of the cluster are not probable candidate for outlier and we can prune out such points from each cluster. Next we calculate a distance based outlier score for remaining points. The computations needed to calculate the outlier score reduces considerably due to the pruning of some points. Based on the outlier score we declare the top n points with the highest score as outliers. The experimental results using real data set demonstrate that even though the number of computations is less, the proposed method performs better than the existing method.

An experimental analysis of outliers detection on static exaustive datasets

“…Clustering methodology requires the definition of a similar properties between patterns, which is not easy to specify without having any prior knowledge about cluster shapes [8]. Clustering is not a new concept but data clustering specifically for outlier detection is a recent discipline under continuous development.…”

Comparative Study of Outlier Detection Algorithms

“…of data used, size of the dataset etc [33]. Now with the tremendous growth of data, best outlier detection algorithms have to be applied on large data sets [8].…”

A systematic review of data quality issues in knowledge discovery tasks

“…Unsupervised learning Supervised learning Statistical Others Figure 5 depicts the techniques to solve the outliers issue. Several papers make frequent use of unsupervised learning (i.e., partitional, density and hierarchical algorithms) and statistical methods [19][20][21][22][23][24]; lesser extent the supervised learning (i.e., variations of decision tree, k-nn and support vector machine algorithms) and genetic algorithms [25][26][27]. …”