Introduction Big Data is the data that are difficult to store, manage, and analyze using traditional database and software techniques. Big Data includes high volume and velocity, and also variety of data that needs for new techniques to deal with it. Intrusion detection system (IDS) is hardware or software monitor that analyzes data to detect any attack toward a system or a network. Traditional intrusion detection system techniques make the system more complex and less efficient when dealing with Big Data, because its analysis properties process is complex and take a long time. The long time it takes to analyze the data makes the system prone to harms for some period of time before getting any alert [1, 2]. Therefore, using Big Data tools and techniques to analyze and store data in intrusion detection system can reduce computation and training time. The IDS has three methods for detecting attacks; Signature-based detection, Anomaly-based detection, and Hybrid-based detection. The signature-based detection is designed to detect known attacks by using signatures of those attacks. It is an effective method of detecting known attacks that are preloaded in the IDS database. Therefore, it is often considered to be much more accurate at identifying an intrusion attempt of Abstract Recently, the huge amounts of data and its incremental increase have changed the importance of information security and data analysis systems for Big Data. Intrusion detection system (IDS) is a system that monitors and analyzes data to detect any intrusion in the system or network. High volume, variety and high speed of data generated in the network have made the data analysis process to detect attacks by traditional techniques very difficult. Big Data techniques are used in IDS to deal with Big Data for accurate and efficient data analysis process. This paper introduced Spark-Chi-SVM model for intrusion detection. In this model, we have used ChiSqSelector for feature selection, and built an intrusion detection model by using support vector machine (SVM) classifier on Apache Spark Big Data platform. We used KDD99 to train and test the model. In the experiment, we introduced a comparison between Chi-SVM classifier and Chi-Logistic Regression classifier. The results of the experiment showed that Spark-Chi-SVM model has high performance, reduces the training time and is efficient for Big Data.

The enormous increase in the use of the Internet in daily life has provided an opportunity for the intruder attempt to compromise the security principles of availability, confidentiality, and integrity. As a result, organizations are working to increase the level of security by using attack detection techniques such as Network Intrusion Detection System (NIDS), which monitors and analyzes network flow and attacks detection. There are a lot of researches proposed to develop the NIDS and depend on the dataset for the evaluation. Datasets allow evaluating the ability in detecting intrusion behavior. This paper introduces a detailed analysis of benchmark and recent datasets for NIDS. Specifically, we describe eight well-known datasets that include: KDD99, NSL-KDD, KYOTO 2006+, ISCX2012, UNSW-NB 15, CIDDS-001, CICIDS2017, and CSE-CIC-IDS2018. For each dataset, we provide a detailed analysis of its instances, features, classes, and the nature of the features. The main objective of this paper is to offer overviews of the datasets are available for the NIDS and what each dataset is comprised of. Furthermore, some recommendations were made to use network-based datasets.

No abstract