Apache Spark and Deep Learning Models for High-Performance Network Intrusion Detection Using CSE-CIC-IDS2018

Hagar, Abdulnaser A.; Gawali, Bharti

doi:10.1155/2022/3131153

Cited by 18 publications

(21 citation statements)

References 41 publications

(42 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the dataset was standard, model training and testing time were also more 26 . Furthermore, the high‐performance NIDS was developed by authors 3 using deep learning models such as CNN and LSTM on CSE‐CICIDSS2018 datasets with 97.08 and 97.70%, respectively. The authors proposed the Apache spark model and achieved 100% accuracy 3 .…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, the high‐performance NIDS was developed by authors 3 using deep learning models such as CNN and LSTM on CSE‐CICIDSS2018 datasets with 97.08 and 97.70%, respectively. The authors proposed the Apache spark model and achieved 100% accuracy 3 . The study 12 suggested the hybrid deep learning strategy using CSE‐CICIDS2018 and DARPA‐IDS datasets and achieved 58.52% and 97.22% accuracy for CNN and mixed approach, respectively.…”

Section: Related Workmentioning

confidence: 99%

“…In this case, computer networks are essential in several information‐sharing domains 1,2 . Thus, network security is paramount due to the growing3 use of computer networks in several domains and applications 4,5 . Several organizations protect their information against network attacks using anti‐viruses, encryption‐decryption, firewalls, and anti‐spam methods 5 .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An LSTM‐based novel near‐real‐time multiclass network intrusion detection system for complex cloud environments

Vibhute,

Khan,

Kanade

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

SummaryThe Internet is connected with everyone for sharing and monitoring digital information. However, securing network resources from malicious activities is critical for several applications. Numerous studies have recently used deep learning‐based models in detecting intrusions and received relatively robust recognition outcomes. Nevertheless, most investigations have operated old datasets, so they could not detect the most delinquent attack information. Therefore, the current research proposes the long short‐term memory (LSTM)‐based near real‐time multiclass network intrusion detection system (NIDS) utilizing complex cloud CSE‐CICIDSS2018 datasets to secure and detect the network anomalous. The proposed strategy utilizes a random forest algorithm for dimensionality reduction and feature selection. In addition, the selected best suitable features were used in a deep learning‐based LSTM model developed for detecting network intrusions. The experimental outcomes reveal that the presented LSTM model obtained 99.66% testing accuracy with 0.12% loss. Thus, the suggested approach can detect network intrusions with the highest precision and lowest rate over the earlier designs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An LSTM‐based novel near‐real‐time multiclass network intrusion detection system for complex cloud environments

Vibhute,

Khan,

Kanade

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Hagar et al [7] proposed a deep learning model using Apache Spark for network intrusion detection to achieve high performance. In comparison, Apache Spark represents an improvisation of accurate prediction.…”

Section: Apache Sparksmentioning

confidence: 99%

“…However, the performance of different activities or services can vary significantly over time, making it difficult to adopt appropriate resource allocations and decisions. [6][7].…”

Section: Introductionmentioning

confidence: 99%

Scheduling of Jobs Allocation for Apache Spark Using Kubernetes for Efficient Execution of Big Data Application

2023

IJIES

View full text Add to dashboard Cite

The use of cloud services is in high demand due to their high storage and computing capacity. Apache spark provides an open deployment framework for data storage and computation using cluster computing. The specified spark core scheduler uses FIFO to manage job execution in batches. However, it may not be suitable for large-scale clusters due to the unevenness in managing resource allocation between different types of applications. Because of this, most of the work executors are still underutilized and resources are wasted in all the pods, leading to cost inefficiencies. Using apache spark on kubernetes to run cloud applications will ensure rapid resource management for workload execution. Incoming workloads vary widely across applications, so it is critical to manage workload allocation to ensure QoS and cost efficiency. This paper proposes a job scheduling mechanism (JSM) for apache spark on kubernetes to dynamically schedule job allocation for the efficient execution of various big data applications. The JSM process predicts the cluster load and suggests relocation of workloads to efficiently distribute the workload to the lower loaded pods in a standard cluster to optimize cost performance. It identifies the upcoming workload of a job and determines the best-fit pod and aims to reduce the usage of CPU and memory which result in enhancing cost efficiency. The JSM's effective management of job allocation and migration among the underload pods preserves the resource and enhances cost efficiency. Experimental settings are configured to evaluate cluster resources with benchmark statistics output for application job execution. The outcome results of cost, job performance, and scheduling overhead show improved cost efficiency for job execution. The comparison with the existing scheduler with varying request load shows an improvisation of 2% in cost efficiency and 3% lower scheduling overhead.

show abstract

Ensemble random forest and deep convolutional neural networks in detecting and classifying the multiple intrusions from near real‐time cloud datasets

Khan,

Haroon

2024

Security and Privacy

View full text Add to dashboard Cite

Due to rapidly growing Internet facilities, intruders can steal and misuse the data saved and stored digitally. In this case, securing digital data is challenging but prominent for various purposes. However, the traditional techniques are insufficient to secure these computer networks and cloud information with a 100% success rate. Recently, machine‐ or deep‐learning‐enabled methods have been used to secure network information, but with some limits. Therefore, the study emphasizes detecting and classifying network intrusion using the proposed ensemble and deep learning models. In this case, we developed the ensemble learning‐enabled random forest algorithm and deep learning‐enabled deep convolutional neural network (CNN) models for securing near real‐time cloud information and designed the intrusion detection system accordingly. The complex and high‐volume CSE‐CICIDS2018 datasets were used to test the developed model in Python programming language implemented with several Python libraries. The outcome of the proposed models indicates that the developed models are promising in securing the cloud information with 97.73% and 99.91% accuracies via ensemble‐random forest and deep CNN models. Thus, the present study models can be applied to other real‐time datasets and computer networks to detect cyber threats effectively.

show abstract

Apache Spark and Deep Learning Models for High-Performance Network Intrusion Detection Using CSE-CIC-IDS2018

Cited by 18 publications

References 41 publications

An LSTM‐based novel near‐real‐time multiclass network intrusion detection system for complex cloud environments

An LSTM‐based novel near‐real‐time multiclass network intrusion detection system for complex cloud environments

Scheduling of Jobs Allocation for Apache Spark Using Kubernetes for Efficient Execution of Big Data Application

Ensemble random forest and deep convolutional neural networks in detecting and classifying the multiple intrusions from near real‐time cloud datasets

Contact Info

Product

Resources

About