Recent Progress of Anomaly Detection

Xu, Xinglian; Liu, Huawen; Yao, Minghai

doi:10.1155/2019/2686378

Cited by 60 publications

(24 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Figure 12 shows the effect of the number of clusters on the TPR and FPR of the OFCOD. The figure shows that the TPR is first enhanced by increasing the number of clusters until reaching its maximum value at k = [4,8] then it tends to decrease again. On the other side, the FPR decreases with the increase of the number of clusters.…”

Section: The Online Stage Of the Proposed Frameworkmentioning

confidence: 99%

“…Outlier detection algorithms have extensively been tackled in the past fifteen years. Many algorithms with different approaches have been introduced in the literature [4][5][6][7][8][9][10][11] which can be, in general, categorized into [12][13][14]: statistical-based [15,16], distance-based [17,18], density-based [19,20] and clustering-based methods [9,[21][22][23]. Statistical-based approaches aim at finding the probability distribution/model of the underlying normal data and define outliers as those points that do not conform to that model.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

OFCOD: On the Fly Clustering Based Outlier Detection Framework

Elmogy

Rizk

Sarhan

2020

Data

View full text Add to dashboard Cite

In data mining, outlier detection is a major challenge as it has an important role in many applications such as medical data, image processing, fraud detection, intrusion detection, and so forth. An extensive variety of clustering based approaches have been developed to detect outliers. However they are by nature time consuming which restrict their utilization with real-time applications. Furthermore, outlier detection requests are handled one at a time, which means that each request is initiated individually with a particular set of parameters. In this paper, the first clustering based outlier detection framework, (On the Fly Clustering Based Outlier Detection (OFCOD)) is presented. OFCOD enables analysts to effectively find out outliers on time with request even within huge datasets. The proposed framework has been tested and evaluated using two real world datasets with different features and applications; one with 699 records, and another with five millions records. The experimental results show that the performance of the proposed framework outperforms other existing approaches while considering several evaluation metrics.

show abstract

Section: The Online Stage Of the Proposed Frameworkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

OFCOD: On the Fly Clustering Based Outlier Detection Framework

Elmogy

Rizk

Sarhan

2020

Data

View full text Add to dashboard Cite

show abstract

“…Aggarwal [7] (2013) reviews the techniques in the literature for outlier ensembles and the principles underlying them. Xu et al [8] (2019) provide a more recent review on the progress made in anomaly detection with a focus to high-dimensional and mixed types. On a side topic, Längkvist et al [9] (2014) present a more general review on unsupervised machine learning applied to time series.…”

Section: Previous Surveys On Anomaly Detectionmentioning

confidence: 99%

Recent Advances in Anomaly Detection Methods Applied to Aviation

2019

View full text Add to dashboard Cite

Anomaly detection is an active area of research with numerous methods and applications. This survey reviews the state-of-the-art of data-driven anomaly detection techniques and their application to the aviation domain. After a brief introduction to the main traditional data-driven methods for anomaly detection, we review the recent advances in the area of neural networks, deep learning and temporal-logic based learning. In particular, we cover unsupervised techniques applicable to time series data because of their relevance to the aviation domain, where the lack of labeled data is the most usual case, and the nature of flight trajectories and sensor data is sequential, or temporal. The advantages and disadvantages of each method are presented in terms of computational efficiency and detection efficacy. The second part of the survey explores the application of anomaly detection techniques to aviation and their contributions to the improvement of the safety and performance of flight operations and aviation systems. As far as we know, some of the presented methods have not yet found an application in the aviation domain. We review applications ranging from the identification of significant operational events in air traffic operations to the prediction of potential aviation system failures for predictive maintenance.

show abstract

“…Anomaly detection refers to the problem of identifying anomalous patterns in data, and the deployment of detectors to KPIs (key performance indicators) metrics of a distributed system has been widely adopted by many Internet companies [4][5][6]. For interesting researchers, we recommend review papers by Chandola et al, [7], Agrawal [8], and Xu et al [9].…”

Section: Introductionmentioning

confidence: 99%

A Two-Layer Architecture for Failure Prediction Based on High-Dimension Monitoring Sequences

Wang

Liu

Feng³

et al. 2021

Complexity

View full text Add to dashboard Cite

In recent years, the distributed architecture has been widely adopted by security companies with the rapid expansion of their business. A distributed system is comprised of many computing nodes of different components which are connected by high-speed communication networks. With the increasing functionality and complexity of the systems, failures of nodes are inevitable which may result in considerable loss. In order to identify anomalies of the possible failures and enable DevOps engineers to operate in advance, this paper proposes a two-layer prediction architecture based on the monitoring sequences of nodes status. Generally speaking, in the first layer, we make use of EXPoSE anomaly detection technique to derive anomaly scores in constant time which are then used as input data for ensemble learning in the second layer. Experiments are conducted on the data provided by one of the largest security companies, and the results demonstrate the predictability of the proposed approach.

show abstract

Recent Progress of Anomaly Detection

Cited by 60 publications

References 50 publications

OFCOD: On the Fly Clustering Based Outlier Detection Framework

OFCOD: On the Fly Clustering Based Outlier Detection Framework

Recent Advances in Anomaly Detection Methods Applied to Aviation

A Two-Layer Architecture for Failure Prediction Based on High-Dimension Monitoring Sequences

Contact Info

Product

Resources

About