A Comparison of Unsupervised Learning Algorithms for Intrusion Detection in IEC 104 SCADA Protocol

Abstract

The power grid is a build-up of a mesh of thousands of sensors, embedded devices, and terminal units that communicate over different media. The heterogeneity of modern and legacy equipment calls for attention towards diverse network security measures. The critical infrastructure employs different security measures to detect and prevent adversaries, e.g., through signature-based tools. These approaches lack the potential to identify unknown attacks. Machine learning has the prospective to address novel attack vectors. This paper systematically evaluates the effcacy of learning algorithms from different families for intrusion detection in IEC 60870-5-104 protocol. One-class SVM and k-Nearest Neighbour unsupervised learning models show small potential when being tested on the IEC 104 unseen dataset with Area Under the Curve score 0.64 and 0.59, in the same order; and Matthews Correlation Coeffcient value 0.3 and 0.2, respectively. The experimental results suggest little feasibility of the evaluated unsupervised learning approaches for anomaly detection in IEC 104 communication and recommend coupling it with other anomaly detection techniques.

Improving anomaly detection in SCADA network communication with attribute extension

“…The supervised and semi-supervised machine learning methods for IEC 104 SCADA protocol outperformed signature-based intrusion detection, and unsupervised learning (Egger et al 2020). Later, systematic performance evaluation of IEC 104 anomaly detection with unsupervised learning approaches was accomplished in Anwar et al (2021). Both studies (Egger et al 2020;Anwar et al 2021), utilised the same IEC 104 dataset.…”

“…The learning algorithm is also an acknowledged choice for intrusion detection in the SCADA network (Rakas et al 2020). Furthermore, recent works on standard SCADA-specific protocol (IEC 104) relayed the algorithm's stable performance for detecting different attacks (Egger et al 2020;Anwar et al 2021). Egger et al (2020) compared intrusion detection of the signature-based method with machine learning methods.…”

“…Supervised and semisupervised (with one-class SVM) learning performed better intrusion detection, while Snort signature-based gave worse (Egger et al 2020). The same protocol dataset is systematically evaluated with other learning algorithms in Anwar et al (2021). Mahwish et al evaluated the SCADA network intrusion detection ability of distance-based, density-based, and kernel-based learning methods in an unsupervised setting for IEC 104 communication protocol.…”

“…The comparison of detection methods revealed that on average one-class SVM method performs steadily for the given SCADA protocol data in reference to other candidate learning methods. In the current work, we draw a comparison with study (Anwar et al 2021). Realising the predictable and steady performance of one-class SVM for SCADA protocol and its ability to segregate communication network data, we intend to amplify the outlier detection capability of one-class SVM for the IEC 104 protocol.…”

On specification-based cyber-attack detection in smart grids

“…In the area of statistically based anomaly detection on IEC-104, the work in [38] presents a 3-value detection method that independently compares the number of packets transmitted in three consecutive time windows against a statistical profile and reports anomalies when a deviation from the specified range is detected. To address the problem of missing labeled data, the work of [39] explores the use of unsupervised machine learning on IEC-104, in particular, one-class support vector machines, isolation forest, histogram-based outlier detection, and k-nearest neighbor are investigated.…”