Overview of Anomaly Detection techniques in Machine Learning

Toshniwal, Akanksha; Mahesh, Kavi; Jayashree, R.

doi:10.1109/i-smac49090.2020.9243329

Cited by 29 publications

(24 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The recall is also very high (it is actually perfect for cryptomining execution detection), while the FPR has a relatively low value of 0.014 for both of the evaluated attacks. For benchmarking, we compared our two-step method with a variety of classical one-step algorithms widely used for anomaly and novelty detection in general [42], [87]- [90], and for IoT attack detection in particular [19], [91]. Two of these algorithms are the building blocks of CADeSH (namely, AE and k-means clustering, which we used to implement F ilter m 1 and F ilter m 2 , respectively).…”

Section: Overall Results and Benchmarkingmentioning

confidence: 99%

CADeSH: Collaborative Anomaly Detection for Smart Homes

Avraham

Libhaber

Shabtai

2023

IEEE Internet Things J.

View full text Add to dashboard Cite

Although home IoT (Internet of Things) devices are typically plain and task oriented, the context of their daily use may affect their traffic patterns. That is, a given IoT device will probably not generate the exact same traffic data when operated by different people in different environments and when connected to different networks with different topologies and communication components. For this reason, anomaly-based intrusion detection systems tend to suffer from a high false positive rate (FPR). To overcome this, we propose a two-step collaborative anomaly detection method which first uses an autoencoder to differentiate frequent ('benign') and infrequent (possibly 'malicious') traffic flows. Clustering is then used to analyze only the infrequent flows and classify them as either known ('rare yet benign') or unknown ('malicious'). Our method is collaborative, in that (1) normal behaviors are characterized more robustly, as they take into account a variety of user interactions and network topologies, and (2) several features are computed based on a pool of identical devices rather than just the inspected device.We evaluated our method empirically, using 21 days of real-world traffic data that emanated from eight identical IoT devices deployed on various networks, one of which was located in our controlled lab where we implemented two popular IoT-related cyber-attacks. Our collaborative anomaly detection method achieved a macro-average area under the precision-recall curve of 0.841, an F1 score of 0.929, and an FPR of only 0.014. These promising results were obtained by using labeled traffic data from our lab as the test set, while training the models on the traffic of devices deployed outside the lab, and thus demonstrate a high level of generalizability. In addition to its high generalizability and promising performance, our proposed method also offers benefits such as privacy preservation, resource savings, and model poisoning mitigation. On top of that, as a contribution to the scientific community, our novel dataset is available online.

show abstract

Section: Overall Results and Benchmarkingmentioning

confidence: 99%

CADeSH: Collaborative Anomaly Detection for Smart Homes

Avraham

Libhaber

Shabtai

2023

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…Application Focus [6] cloud computing IDS [5] computer networks IDS [24] malicious actor space and information networks [2] IIoT GNNs [23] attention-based AD - [4] attention-based AD - [1] non-IoT graph-based AD deep learning [3] general AD -…”

Section: Referencementioning

confidence: 99%

A Survey of AI-Based Anomaly Detection in IoT and Sensor Networks

DeMedeiros

Hendawi

Álvarez

2023

Sensors

View full text Add to dashboard Cite

Machine learning (ML) and deep learning (DL), in particular, are common tools for anomaly detection (AD). With the rapid increase in the number of Internet-connected devices, the growing desire for Internet of Things (IoT) devices in the home, on our person, and in our vehicles, and the transition to smart infrastructure and the Industrial IoT (IIoT), anomaly detection in these devices is critical. This paper is a survey of anomaly detection in sensor networks/the IoT. This paper defines what an anomaly is and surveys multiple sources based on those definitions. The goal of this survey was to highlight how anomaly detection is being performed on the Internet of Things and sensor networks, identify anomaly detection approaches, and outlines gaps in the research in this domain.

show abstract

“…There are contradictory conclusions made on the effectiveness of the resembling technique for solving the class imbalance issue. Supervised machine learning techniques failed to detect unseen or new types of anomalies while unsupervised machine learning techniques tend to classify noises as anomalies [21], [22]. Thus, two variations of hybrid models which combine both the supervised and unsupervised techniques are proposed so that it can exceed the performance of conventional machine learning techniques in anomaly detection.…”

Section: Literature Reviewmentioning

confidence: 99%

Hybrid machine learning approach for anomaly detection

Lok

Hameed

Rana

2022

IJEECS

View full text Add to dashboard Cite

This research aims to <span lang="EN-US">improve anomaly detection performance by developing two variants of hybrid models combining supervised and unsupervised machine learning techniques. Supervised models cannot detect new or unseen types of anomaly. Hence in variant 1, a supervised model that detects normal samples is followed by an unsupervised learning model to screen anomaly. The unsupervised model is weak in differentiating between noise and fraud. Hence in variant 2, the hybrid model incorporates an unsupervised model that detects anomaly is followed by a supervised model to validate an anomaly. Three different datasets are used for model evaluation. The experiment is begun with 5 supervised models and 3 unsupervised models. After performance evaluation, 2 supervised models with the highest F1-Score and one unsupervised model with the best recall value are selected for hybrid model development. The variant 1 hybrid model recorded the best recall value across all the experiments, indicating that it is the best at detecting actual fraud and less likely to miss it compared to other models. The variant 2 hybrid model can improve the precision score significantly compared to the original unsupervised model, indicating that it is better in separating noise from fraud,</span>

show abstract

Overview of Anomaly Detection techniques in Machine Learning

Cited by 29 publications

References 52 publications

CADeSH: Collaborative Anomaly Detection for Smart Homes

CADeSH: Collaborative Anomaly Detection for Smart Homes

A Survey of AI-Based Anomaly Detection in IoT and Sensor Networks

Hybrid machine learning approach for anomaly detection

Contact Info

Product

Resources

About