A Dynamic Ensemble Learning Framework for Data Stream Analysis and Real-Time Threat Detection

Demertzis, Konstantinos; Iliadis, Lazaros; Anezakis, Vardis-Dimitris

doi:10.1007/978-3-030-01418-6_66

Cited by 10 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In essence, the information ecosystem and the importance of its applications require the creation of a cybersecurity environment with fully automated solutions. These solutions include real-time incident handling, analysis, and other security information to identify known and unknown threats and reduce the risk for the critical data through a scalable troubleshooting and logging approach [10], [11].…”

Section: Introductionmentioning

confidence: 99%

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

Self Cite

View full text Add to dashboard Cite

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real time. Based on a weight agnostic neural networks (WANNs) methodology, we propose an automated searching neural net architecture strategy that can perform various tasks such as identifying zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

show abstract

Section: Introductionmentioning

confidence: 99%

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, it would be important to study the extension of this system by implementing more complex architectures with Siamese neural networks in parallel and distributed real time data stream environments [45].…”

Section: Discussionmentioning

confidence: 99%

GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification

Demertzis

Iliadis

2020

Algorithms

Self Cite

View full text Add to dashboard Cite

Deep learning architectures are the most effective methods for analyzing and classifying Ultra-Spectral Images (USI). However, effective training of a Deep Learning (DL) gradient classifier aiming to achieve high classification accuracy, is extremely costly and time-consuming. It requires huge datasets with hundreds or thousands of labeled specimens from expert scientists. This research exploits the MAML++ algorithm in order to introduce the Model-Agnostic Meta-Ensemble Zero-shot Learning (MAME-ZsL) approach. The MAME-ZsL overcomes the above difficulties, and it can be used as a powerful model to perform Hyperspectral Image Analysis (HIA). It is a novel optimization-based Meta-Ensemble Learning architecture, following a Zero-shot Learning (ZsL) prototype. To the best of our knowledge it is introduced to the literature for the first time. It facilitates learning of specialized techniques for the extraction of user-mediated representations, in complex Deep Learning architectures. Moreover, it leverages the use of first and second-order derivatives as pre-training methods. It enhances learning of features which do not cause issues of exploding or diminishing gradients; thus, it avoids potential overfitting. Moreover, it significantly reduces computational cost and training time, and it offers an improved training stability, high generalization performance and remarkable classification accuracy. Algorithms 2020, 13, 61 2 of 25 Algorithms 2020, 13, 61 3 of 25In contrast to the learning process of the traditional neural networks, it is not sufficient for the network to learn good representations of the training classes, as the testing classes are distinct and they are not presented in training. However, it is desirable to learn features which distinguish the existing classes.The evaluation process consists of two distinct stages of the following format [4]:Step 1: Given k examples (value of k-shot), if k = 1, then the process is called one-shot; if k = 5, five-shot, and so on. The parameter k represents the number of labeled samples given to the algorithm by each class. By considering these samples, which comprise the support set, the network is required to classify and eventually adapt to existing classes.Step 2: Unknown examples of the labeled classes are presented randomly, unlike the ones presented in the previous step, which the network is called to correctly classify. The set of examples in this stage is known as the query set.The above procedure (steps) is repeated many times using random classes and examples which are sampled from the testing-evaluation set.As it is immediately apparent from the description of the evaluation process, as the number of classes increases, the task becomes more difficult, because the network has to decide between several alternatives. This means that Zero-shot Learning [5] is clearly more difficult than the one-shot, which is more difficult than the five-shot, and so on. Although humans have the ability to cope with this process, traditional ANN require many more examples to genera...

show abstract

“…This study has emerged after extensive and long-term research on stream analysis, and it performs actions on real-time data. This paper exploits and considers some of the most important suggestions and innovations of our prior research [27][28][29][30][31].…”

Section: Methodsmentioning

confidence: 99%

Identifying data streams anomalies by evolving spiking restricted Boltzmann machines

Xing

Demertzis

Yang

2019

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

Data streams are characterized by high volatility, and they drastically change in an unpredictable way over time. In the typical case, newer data are the most important, as the concept of aging is based on their timing. These flows require realtime processing in order to extract meaningful information that will allow for essential and targeted responses to changing circumstances. Knowledge mining is a real-time process performed on a subset of the data streams, which contains a small but recent part of the observations. Timely security requirements call for further quest of optimal approaches, capable of improving the reliability and the accuracy of the employed classifiers. This research introduces a real-time evolving spiking restricted Boltzmann machine approach, for efficient anomaly detection in data streams. Testing has proved that the proposed algorithm maximizes the classification accuracy and at the same time minimizes the computational resources requirements. A comparative analysis has shown that it outperforms other data flow analysis algorithms.Keywords Big Data Á Data streams analysis Á Evolving spiking neural networks Á Restricted Boltzmann machines Á Deep learning Á Real-time anomaly detection

show abstract

A Dynamic Ensemble Learning Framework for Data Stream Analysis and Real-Time Threat Detection

Cited by 10 publications

References 29 publications

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification

Identifying data streams anomalies by evolving spiking restricted Boltzmann machines

Contact Info

Product

Resources

About