Robust Anomaly Detection on Unreliable Data

Abstract

Classification algorithms have been widely adopted to detect anomalies for various systems, e.g., IoT, cloud and face recognition, under the common assumption that the data source is clean, i.e., features and labels are correctly set. However, data collected from the wild can be unreliable due to careless annotations or malicious data transformation for incorrect anomaly detection. In this paper, we present a two-layer on-line learning framework for robust anomaly detection (RAD) in the presence of unreliable anomaly labels, where the first layer is to filter out the suspicious data, and the second layer detects the anomaly patterns from the remaining data. To adapt to the on-line nature of anomaly detection, we extend RAD with additional features of repetitively cleaning, conflicting opinions of classifiers, and oracle knowledge. We on-line learn from the incoming data streams and continuously cleanse the data, so as to adapt to the increasing learning capacity from the larger accumulated data set. Moreover, we explore the concept of oracle learning that provides additional information of true labels for difficult data points. We specifically focus on three use cases, (i) detecting 10 classes of IoT attacks, (ii) predicting 4 classes of task failures of big data jobs, (iii) recognising 20 celebrities faces. Our evaluation results show that RAD can robustly improve the accuracy of anomaly detection, to reach up to 98% for IoT device attacks (i.e., +11%), up to 84% for cloud task failures (i.e., +20%) under 40% noise, and up to 74% for face recognition (i.e., +28%) under 30% noisy labels. The proposed RAD is general and can be applied to different anomaly detection algorithms.

Smart City Ranking System: A Supporting Tool to Manage Migration Trends for Australian Cities

“…That makes the availability, reliance, and accuracy of data as a pivotal factor in smart city ranking system. With the high reliance of data needed for each city, the collection of this information may seem unreliable due to the "careless annotation or malicious data" of the sources [76]. As a result, this limits the number of cities and indicators used for the ranking system.…”

Leveraging Expert Knowledge for Label Noise Mitigation in Machine Learning

“…The most beneficial advantage of this framework is that it facilitates to create new algorithms that combine the strength of each part. Outlier detection based algorithms detect if the label noise sample is corresponding to an outlier sample, whose anomaly score exceeds a predefined threshold [15,16]. Many algorithms use a model that trained from the original training set to classify the training set itself to detect the label noise [17,18].…”

Autonomic rejuvenation of cloud applications as a countermeasure to software anomalies

“…A different research line has dealt with the construction of reliable prediction models when the data to be used for training is unreliable 44 or when bias and variance in the data could cause inaccurate prediction 45 . This important research line is orthogonal to the one carried in this article.…”