OFCOD: On the Fly Clustering Based Outlier Detection Framework

Abstract

In data mining, outlier detection is a major challenge as it has an important role in many applications such as medical data, image processing, fraud detection, intrusion detection, and so forth. An extensive variety of clustering based approaches have been developed to detect outliers. However they are by nature time consuming which restrict their utilization with real-time applications. Furthermore, outlier detection requests are handled one at a time, which means that each request is initiated individually with a particular set of parameters. In this paper, the first clustering based outlier detection framework, (On the Fly Clustering Based Outlier Detection (OFCOD)) is presented. OFCOD enables analysts to effectively find out outliers on time with request even within huge datasets. The proposed framework has been tested and evaluated using two real world datasets with different features and applications; one with 699 records, and another with five millions records. The experimental results show that the performance of the proposed framework outperforms other existing approaches while considering several evaluation metrics.

Data Mining in Internet of Things Systems: A Literature Review

“…Usually, clustering techniques are designed for a specific application for which they perform well. Examples of clustering algorithms are: Kmean, K-Nearest Neighbor, K-medoids, hierarchical clustering (CURE, SVD, ROCK, BIRCH) [1], Density based clustering (DBSCAN, OPTICS, DENCLUE), Grid based clustering (STRING, WaveCluster [2,[56][57][58][59][60]65]. With the development of sensor technologies, IoT and WSN create user environments that are intelligent enough to recognize user activity and respond appropriately.…”

“…The goal of density-based partitioning techniques is to find low-dimensional, densely connected data, also referred to as spatial data. DBSCAN (Density Based Spatial Clustering of Applications with Noise) [2] is one of the related studies. Hierarchical agglomeration is one of the processing phases used by gridbased partitioning algorithms, which also perform space segmentation.…”

“…Anomalies or outliers, which are unexpectedly useful pieces of information present in raw data, can occasionally cause data mining methods to fail [2,8,25]. The characteristics of outlier data objects differ significantly from those of typical data objects.…”

Drone-Based Water Level Detection in Flood Disasters

“…This is done by calculating the standard deviation of the image represents the difference between high pass filtered versions of both the original image and its low pass filtered one [ 34 ]. Images with high blurring factors or with low resolution are excluded as they may deceive the annotation process and thus the trained model [ 35 , 36 ].…”