Towards scalable rough set based attribute subset selection for intrusion detection using parallel genetic algorithm in MapReduce

El-Alfy, El-Sayed M.; Alshammari, Mashaan

doi:10.1016/j.simpat.2016.01.010

Cited by 40 publications

(18 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A hybrid feature selection method with GA wrapper using mutual information and using SVM is proposed by Huang et al [24]. Some recent attempts to improve the optimised feature selection process by parallel processing are: [8], [25], [26], [27].…”

Section: Existing Workmentioning

confidence: 99%

Information gain directed genetic algorithm wrapper feature selection for credit rating

Jadhav

Jenkins

2018

Applied Soft Computing

241

View full text Add to dashboard Cite

Financial credit scoring is one of the most crucial processes in the finance industry sector to be able to assess the credit-worthiness of individuals and enterprises. Various statistics-based machine learning techniques have been employed for this task. "Curse of Dimensionality" is still a significant challenge in machine learning techniques. Some research has been carried out on Feature Selection (FS) using genetic algorithm as wrapper to improve the performance of credit scoring models. However, the challenge lies in finding an overall best method in credit scoring problems and improving the time-consuming process of feature selection. In this study, the credit scoring problem is investigated through feature selection to improve classification performance. This work proposes a novel approach to feature selection in credit scoring applications, called as Information Gain Directed Feature Selection algorithm (IGDFS), which performs the ranking of features based on information gain, propagates the top m features through the GA wrapper (GAW) algorithm using

show abstract

Section: Existing Workmentioning

confidence: 99%

Information gain directed genetic algorithm wrapper feature selection for credit rating

Jadhav

Jenkins

2018

Applied Soft Computing

241

View full text Add to dashboard Cite

show abstract

“…In order to overcome these weaknesses, a set of parallel and distributed rough set methods has been proposed in the literature to ensure feature selection but in different contexts. For example, some of these distributed methods adopt some evolutionary algorithms, such as the work proposed in [12], where authors defined a hierarchical MapReduce implementation of a parallel genetic algorithm for determining the minimum rough set reduct, i.e., the set of the selected features. Within another context, the context of limited labeled big data, in [32], authors introduced a theoretic framework called local rough set and developed a series of corresponding concept approximation and attribute reduction algorithms with linear time complexity, which can efficiently and effectively work in limited labeled big data.…”

Section: Literature Reviewmentioning

confidence: 99%

A scalable and effective rough set theory-based approach for big data pre-processing

et al. 2020

View full text Add to dashboard Cite

A big challenge in the knowledge discovery process is to perform data pre-processing, specifically feature selection, on a large amount of data and high dimensional attribute set. A variety of techniques have been proposed in the literature to deal with this challenge with different degrees of success as most of these techniques need further information about the given input data for thresholding, need to specify noise levels or use some feature ranking procedures. To overcome these limitations, rough set theory (RST) can be used to discover the dependency within the data and reduce the number of attributes enclosed in an input data set while using the data alone and requiring no supplementary information. However, when it comes to massive data sets, RST reaches its limits as it is highly computationally expensive. In this paper, we propose a scalable and effective rough set theory-based approach for large-scale data pre-processing, specifically for feature selection, under the Spark framework. In our detailed experiments, data sets with up to 10,000 attributes have been considered, revealing that our proposed solution achieves a good speedup and performs its feature selection task well without sacrificing performance. Thus, making it relevant to big data. Keywords Big data • Data pre-processing • Rough set theory • Distributed processing • Scalability • High-performance computing This work is part of a project that has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 702527.

show abstract

“…Parallel programming [25][26][27][28][29][30]63] is another approach to accelerate the process of knowledge acquisition. Most of the methods in this approach are based on MapReduce.…”

Section: Literature Reviewmentioning

confidence: 99%

“…In [29], Qian et al proposed three parallelization strategies for attribute reduction, namely "Task-Parallelism", "Data-Parallelism", and "Data+Task-Parallelism". El-Alfy et al [63] introduced a parallel genetic algorithm to approximate the minimum reduct and applied it to intrusion detection in computer networks. Li and Chen [27,28] computed set approximations and reducts in parallel using dominance-based neighborhood rough sets, which considers the orders of numerical and categorical attribute values.…”

Section: Literature Reviewmentioning

confidence: 99%

Parallel Computation of Rough Set Approximations in Information Systems with Missing Decision Data

et al. 2018

View full text Add to dashboard Cite

The paper discusses the use of parallel computation to obtain rough set approximations from large-scale information systems where missing data exist in both condition and decision attributes. To date, many studies have focused on missing condition data, but very few have accounted for missing decision data, especially in enlarging datasets. One of the approaches for dealing with missing data in condition attributes is named twofold rough approximations. The paper aims to extend the approach to deal with missing data in the decision attribute. In addition, computing twofold rough approximations is very intensive, thus the approach is not suitable when input datasets are large. We propose parallel algorithms to compute twofold rough approximations in large-scale datasets. Our method is based on MapReduce, a distributed programming model for processing large-scale data. We introduce the original sequential algorithm first and then the parallel version is introduced. Comparison between the two approaches through experiments shows that our proposed parallel algorithms are suitable for and perform efficiently on large-scale datasets that have missing data in condition and decision attributes.

show abstract

Towards scalable rough set based attribute subset selection for intrusion detection using parallel genetic algorithm in MapReduce

Cited by 40 publications

References 27 publications

Information gain directed genetic algorithm wrapper feature selection for credit rating

Information gain directed genetic algorithm wrapper feature selection for credit rating

A scalable and effective rough set theory-based approach for big data pre-processing

Parallel Computation of Rough Set Approximations in Information Systems with Missing Decision Data

Contact Info

Product

Resources

About