Fast outlier detection using rough sets theory

Abstract

In many Knowledge Discovery applications, finding outliers is more interesting than finding inliers in a dataset. The perception of outliers is rare cases in dataset in which is being described as abnormal data in the information table. Outliers detections are applied in many important applications like fraud detection systems to uncover the suspicious objects which may have important knowledge hidden in the system. A new outlier detection technique based on Rough Sets Theory (RST) is hereby proposed. RSetOF is a new measure for the outlier factor based on RST. By employing this factor, a new formulation for detecting outlier is established. The outlyingness of outliers objects in a dataset using this measurement is identified. To detect outliers, two measurements which are the top n ratio and the coverage ratio are presented. Finding top n outliers from all objects allow searching of outliers from top ranked records based on the least outlier factor value. The capability in detecting outliers at top n number of outliers will indicate how fast the detection is. The efficiency of this technique by obtaining the coverage ratio value is then tested. The maximum percentage of coverage obtained shows the maximum number of outliers detected belonging to rare cases. A comparison is hence carried out to examine the performance of the RSetAlg with a selective outlier detection method, the Frequent Pattern method referred to as FindFPOF. Ten benchmark datasets for assessing the outlier detection technique are used for this purpose. The experimental result shows that the proposed technique is competitive and proven to be better in speed of detection than the other technique. The fast and efficient detection of outliers has proven its potential as a new outliers detection technique based on RST.

Deviation detection in text using conceptual graph interchange format and error tolerance dissimilarity function

“…Conventional deviation detection methods developed for structured categorical data as reviewed in [19] are inappropriate for handling unstructured text. The data that are considered in our work is textual data.…”

A Rough set outlier detection based on Particle Swarm Optimization

“…Based on Reduct computation, a non-Reduct of B is defined as, there exist a set of attributes ‫ܤ‬ െ ‫ܤ‬Ԣ ‫ك‬ ‫ܤ‬ , such that all attributes ܽ ‫א‬ ‫ܤ‬Ԣ are indispensable, and ‫ܤ‪ሺ‬ܦܰܫ‬ െ ‫ܤ‬ ᇱ ሻ ് ‫.‪ሻ‬ܤ‪ሺ‬ܦܰܫ‬ Non-Reduct can be defined as a non interesting set of attributes which is presumed to contain outlier's knowledge [8]. A simple DS with distribution of equivalence class is shown in Table1.…”

“…That is, both algorithms used the same outlier detection method which is based on Rough set outlier factor (RSetOF) [8]. The result shown the BinPSO reached lower values in computation time for all 12 data sets, but has higher values for standard deviation.…”

“…Although this method is much better than [5] but it increases the computation complexity. Shaari et al, [7,8] proposed a non-Reduct approach based on Rough set to detect an outlier. This approach find minimal attributes which are indiscern in attribute values as well as indiscern in decision attributes from the matrix.…”