Big Data Analytics and Deep Learning are two high-focus of data science. Big Data has become important as many organizations both public and private have been collecting massive amounts of domain-specific information, which can contain useful information about problems such as national intelligence, cyber security, fraud detection, marketing, and medical informatics. Companies such as Google and Microsoft are analyzing large volumes of data for business analysis and decisions, impacting existing and future technology. Deep Learning algorithms extract high-level, complex abstractions as data representations through a hierarchical learning process. Complex abstractions are learnt at a given level based on relatively simpler abstractions formulated in the preceding level in the hierarchy. A key benefit of Deep Learning is the analysis and learning of massive amounts of unsupervised data, making it a valuable tool for Big Data Analytics where raw data is largely unlabeled and un-categorized. In the present study, we explore how Deep Learning can be utilized for addressing some important problems in Big Data Analytics, including extracting complex patterns from massive volumes of data, semantic indexing, data tagging, fast information retrieval, and simplifying discriminative tasks. We also investigate some aspects of Deep Learning research that need further exploration to incorporate specific challenges introduced by Big Data Analytics, including streaming data, high-dimensional data, scalability of models, and distributed computing. We conclude by presenting insights into relevant future works by posing some questions, including defining data sampling criteria, domain adaptation modeling, defining criteria for obtaining useful data abstractions, improving semantic indexing, semi-supervised learning, and active learning.

No abstract

No abstract

The tremendous growth in computer network and Internet usage, combined with the growing number of attacks makes network security a topic of serious concern. One of the most prevalent network attacks that can threaten computers connected to the network is brute force attack. In this work we investigate the use of machine learners for detecting brute force attacks (on the SSH protocol) at the network level. We base our approach on applying machine learning algorithms on a newly generated dataset based upon network flow data collected at the network level. Applying detection at the network level makes the detection approach more scalable. It also provides protection for the hosts who do not have their own protection. The new dataset consists of real-world network data collected from a production network. We use four different classifiers to build brute force attack detection models. The use of different classifiers facilitates a relatively comprehensive study on the effectiveness of machine learners in the detection of brute force attack on the SSH protocol at the network level. Empirical results show that the machine learners were quite successful in detecting the brute force attacks with a high detection rate and low false alarms. We also investigate the effectiveness of using ports as features during the learning process. We provide a detailed analysis of how the models built can change as a result of including or excluding port features.Keywords -Brute force attack, network flow, network-level attack detection, machine learning.

No abstract

No abstract

Considering the large quantity of the data flowing through the network routers, there is a very high demand to detect malicious and unhealthy network traffic to provide network users with reliable network operation and security of their information. Predictive models should be built to identify whether a network traffic record is healthy or malicious. To build such models, machine learning methods have started to be used for the task of network intrusion detection. Such predictive models must monitor and analyze a large amount of network data in a reasonable amount of time (usually real time). To do so, they cannot always process the whole data and there is a need for data reduction methods, which reduce the amount of data that needs to be processed. Feature selection is one of the data reduction methods that can be used to decrease the process time. It is important to understand which features are most relevant to determining if a network traffic record is malicious and avoid using the whole feature set to make the processing time more efficient. Also it is important that the simple model built from the reduced feature set be as effective as a model which uses all the features. Considering these facts, feature selection is a very important pre-processing step in the detection of network attacks. The goal is to remove irrelevant and redundant features in order to increase the overall effectiveness of an intrusion detection system without negatively affecting the classification performance. Most of the previous feature selection studies in the area of intrusion detection have been applied on the KDD 99 dataset. As KDD 99 is an outdated dataset, in this paper, we compare different feature selection methods on a relatively new dataset, called Kyoto 2006+. There is no comprehensive comparison of different feature selection approaches for this dataset. In the present work, we study four filter-based feature selection methods which are chosen from two categories for the application of network intrusion detection. Three filter-based feature rankers and one filter-based subset evaluation technique are compared together along with the null case which applies no feature selection. We also apply statistical analysis to determine whether performance differences between these feature selection methods are significant or not. We find that among all the feature selection methods, Signal-to-Noise (S2N) gives the best performance results. It also outperforms no feature selection approach in all the experiments.

No abstract