Toward Stream-Based IP Flow Analysis

Jirsík, Tomáš; Čermák, Milan; Tovarňák, Daniel; Čeleda, Pavel

doi:10.1109/mcom.2017.1600972

Cited by 19 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The prototype serves as a visualization of network parameters. Stream4Flow, however, lacks the intelligence to perform anomaly detection. Hogzilla is an intrusion detection system (IDS) with support for Snort, SFlows, GrayLog, Apache Spark, HBase, and libnDPI, which provides network anomaly detection.…”

Section: Related Workmentioning

confidence: 99%

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Lopez

Mattos

Duarte

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

The late detection of security threats causes a significant increase in the risk of irreparable damages and restricts any defense attempt. In this paper, we propose a sCAlable TRAffic Classifier and Analyzer (CATRACA). CATRACA works as an efficient online Intrusion Detection and Prevention System implemented as a Virtualized Network Function. CATRACA is based on Apache Spark, a Big Data Streaming processing system, and it is deployed over the Open Platform for Network Functions Virtualization (OPNFV), providing an accurate real-time threat-detection service. The system presents a friendly graphical interface that provides real-time visualization of the traffic and the attacks that occur in the network. Our prototype can differentiate normal traffic from denial of service (DoS) attacks and vulnerability probes over 95% accuracy under three different datasets. Moreover, CATRACA handles streaming data under concept drift detection with more than 85% of accuracy. KEYWORDSbig data, network traffic classification, stream processing, threat detection, virtual network function INTRODUCTIONThe Internet is facing constant changes, from the diversity of the user, the complexity of its application, until the heterogeneity of the information producers. 1 As a consequence, traffic monitoring, a critical task in maintaining the stability, reliability, and security of computer networks, is facing new challenges. 2 Current network monitoring tools are inadequate for current speed and management needs of large network domains.To ensure network security, new systems must be designed since current security systems such as Security Information and Event Management (SIEM) are inadequate. While 82% of security threats occur in minutes, an intrusion can take up to 8 months to be detected. 3 It is essential that the detection time is the least possible so that intrusion prevention can be effective. 4Security incidents have increased their complexity, and simple analysis and filtering of packets are no longer sufficient. Attackers try to hide malicious traffic from the security tools by forging the source IP and dynamically changing TCP port. In this context, a promising alternative for classifying network traffic and detect threats is to apply Machine Learning (ML) techniques. These techniques are suitable for big data, with more samples to train the classifier, as methods have higher effectiveness. 5 With a large number of features, however, ML techniques perform results with high latency due to computational resource consumption. This high latency is a disadvantage for applications that use machine learning for real-time classification. For example, network monitoring applications must analyze data and detect threats as quickly as possible. In this context, real-time stream processing allows the immediate analysis of different types of data and consequently benefits traffic monitoring for security threat detection. Open source distributed processing platforms such as Apache Storm, 6 Apache Flink, 7 and Apache Spark 8 process big data w...

show abstract

Section: Related Workmentioning

confidence: 99%

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Lopez

Mattos

Duarte

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Alternative methods, which implement hashing techniques (e.g., sketches) on the network hardware [21], or require enhanced programmability (i.e., beyond OpenFlow) of the forwarding plane [22] [23] [24], still have very limited support on devices, which makes their applicability uncertain. At the same time, solutions based on stream processing [40] [39] can pose a much higher processing burden on local managers, e.g., in the case of large-scale networks with a limited number of LMs, and are unsuitable for many management applications due to the adoption of packet sampling [31].…”

Section: B Monitoring Software-defined Networkmentioning

confidence: 99%

Self-Adaptive Decentralized Monitoring in Software-Defined Networks

Tangari

Tuncer

Charalambides

et al. 2018

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

The Software-Defined Networking (SDN) paradigm can allow network management solutions to automatically and frequently reconfigure network resources. When developing SDNbased management architectures, it is of paramount importance to design a monitoring system that can provide timely and consistent updates to heterogeneous management applications. To support such applications operating with low latency requirements, the monitoring system should scale with increasing network size and provide precise network views with minimum overhead on the available resources. In this paper we present a novel, self-adaptive, decentralized framework for resource monitoring in SDN. Our framework enables accurate statistics to be collected with limited burden on the network resources. This is realized through a self-tuning, adaptive monitoring mechanism that automatically adjusts its settings based on the traffic dynamics. We evaluate our proposal based on a realistic use case scenario, where a content distribution service and an on-demand gaming platform are deployed within an ISP network. The results show that reduced monitoring latencies are obtained with the proposed framework, thus enabling shorter reconfiguration control loops. In addition, the proposed adaptive monitoring method achieves significant gain in terms of monitoring overhead, while preserving the performance of the services considered.

show abstract

“…To demonstrate a pilot implementation of the framework, we introduce a publicly available prototype Stream4Flow 1 [33]. To highlight the advantages of the framework, we implement NwCSA framework based on IP flow data source.…”

Section: B Prototype Implementationmentioning

confidence: 99%

Toward real-time network-wide cyber situational awareness

Jirsík

Čeleda

2018

NOMS 2018 - 2018 IEEE/IFIP Network Operations and Management Symposium

Self Cite

View full text Add to dashboard Cite

In today's complex computer networks, we are constantly facing a risk of data loss, system compromise, or intellectual property theft. The complexity of the networks hinders their effective defense. A Network-wide Cyber Situational Awareness (NwCSA) has been introduced to assist a network security administrator with network security. The concept, however, faces several challenges that hinder an efficient application of the NwCSA in a real-world environment. The challenges include the overload of raw data, low speed of reaction, and a lack of context and unified view on a network. In this paper, we present a novel framework that faces above mentioned challenges. The framework leverages a distributed data stream processing system and methods for real-time big data processing. The framework is evaluated with respect to stated requirements on systems for NwCSA. Moreover, we present a prototype framework implementation and provide lessons learned from its real-world deployment.

show abstract

Toward Stream-Based IP Flow Analysis

Cited by 19 publications

References 8 publications

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Self-Adaptive Decentralized Monitoring in Software-Defined Networks

Toward real-time network-wide cyber situational awareness

Contact Info

Product

Resources

About