Network anomaly detection using IP flows with Principal Component Analysis and Ant Colony Optimization

Fernandes, Gilberto; Carvalho, Luiz F.; Rodrigues, Joel J. P. C.; Proença, Mário Lemes

doi:10.1016/j.jnca.2015.11.024

Cited by 61 publications

(26 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ant Colony Optimization is where a population of agents competing and globally asynchronous, cooperate with one another to find an optimal solution. Ant Colonization Optimization and Dynamic Time Wrapping methods have been used in the environment of pattern recognition and anomaly detection (Fernandes, Carvalho, Rodrigues, & Proença, 2016).…”

Section: Ant Colonymentioning

confidence: 99%

Real-time big data processing for anomaly detection: A Survey

Habeeb¹,

Nasaruddin²,

Gani³

et al. 2019

International Journal of Information Management

273

View full text Add to dashboard Cite

The advent of connected devices and omnipresence of Internet have paved way for intruders to attack networks, which leads to cyber-attack, financial loss, information theft in healthcare, and cyber war. Hence, network security analytics has become an important area of concern and has gained intensive attention among researchers, off late, specifically in the domain of anomaly detection in network, which is considered crucial for network security. However, preliminary investigations have revealed that the existing approaches to detect anomalies in network are not effective enough, particularly to detect them in real time. The reason for the inefficacy of current approaches is mainly due the amassment of massive volumes of data though the connected devices. Therefore, it is crucial to propose a framework that effectively handles real time big data processing and detect anomalies in networks. In this regard, this paper attempts to address the issue of detecting anomalies in real time. Respectively, this paper has surveyed the state-of-the-art real-time big data processing technologies related to anomaly detection and the vital characteristics of associated machine learning algorithms. This paper begins with the explanation of essential contexts and taxonomy of real-time big data processing, anomalous detection, and machine learning algorithms, followed by the review of big data processing technologies. Finally, the identified research challenges of real-time big data processing in anomaly detection are discussed.

show abstract

Section: Ant Colonymentioning

confidence: 99%

Real-time big data processing for anomaly detection: A Survey

Habeeb¹,

Nasaruddin²,

Gani³

et al. 2019

International Journal of Information Management

273

View full text Add to dashboard Cite

show abstract

“…Performance is a key factor when trying to utilize anomaly detection techniques and there are many examples where this is apparent. Methods such as Principal component analysis [3], K nearest neighbour [9] and ensemble techniques [10], [11] have been used to various degrees of success in this task.…”

Section: Related Workmentioning

confidence: 99%

“…While much research has been conducted on various methods for anomaly based systems using a variety of approaches [1][2][3][4][5], key limitations apply when attempting to adapt these approaches to a real-time system. These include, most notably, computational cost.…”

Section: Introductionmentioning

confidence: 99%

NetFlow Anomaly Detection Though Parallel Cluster Density Analysis in Continuous Time-Series

Flanagan

Fallon

Connolly

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The increase in malicious network based attacks has resulted in a growing interest in network anomaly detection. The ability to detect unauthorized or malicious activity on a network is of importance to any organization. With the increase in novel attacks, anomaly detection techniques can be more successful in detecting unknown malicious activity in comparison to traditional signature based methods. However, in a real-world environment, there are many variables that cannot be simulated. This paper proposes an architecture where parallel clustering algorithms work concurrently in order to detect abnormalities that may be lost while traversing over time-series windows. The presented results describe the NetFlow activity of the NPD Group, Inc. over a 24-hour period. The presented results contain real-world anomalies that were detected.

show abstract

“…Multivariate Analysis has been recognized as an outstanding approach for anomaly detection in several domains, including industrial monitoring [5] and networking [6]. In the field of industrial processing, a well developed strategy is Multivariate Statistical Process Control (MSPC).…”

Section: Introductionmentioning

confidence: 99%

Semi-Supervised Multivariate Statistical Network Monitoring for Learning Security Threats

Camacho

Maciá‐Fernández

Fuentes-Garcia

et al. 2019

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

This paper presents a semi-supervised approach for intrusion detection. The method extends the unsupervised Multivariate Statistical Network Monitoring approach based on Principal Component Analysis by introducing a supervised optimization technique to learn the optimum scaling in the input data. It inherits the advantages of the unsupervised strategy, capable of uncovering new threats, with that of supervised strategies, able of learning the pattern of a targeted threat. The supervised learning is based on an extension of the gradient descent method based on Partial Least Squares (PLS). Moreover, we enhance this method by using sparse PLS variants. The practical application of the system is demonstrated on a recently published real case study, showing relevant improvements in detection performance and in the interpretation of the attacks.

show abstract

Network anomaly detection using IP flows with Principal Component Analysis and Ant Colony Optimization

Cited by 61 publications

References 29 publications

Real-time big data processing for anomaly detection: A Survey

Real-time big data processing for anomaly detection: A Survey

NetFlow Anomaly Detection Though Parallel Cluster Density Analysis in Continuous Time-Series

Semi-Supervised Multivariate Statistical Network Monitoring for Learning Security Threats

Contact Info

Product

Resources

About