Modified Genetic Algorithm for Feature Selection and Hyper Parameter Optimization: Case of XGBoost in Spam Prediction

Abstract

Recently, spam on online social networks has attracted attention in the research and business world. Twitter has become the preferred medium to spread spam content. Many research efforts attempted to encounter social networks spam. Twitter brought extra challenges represented by the feature space size, and imbalanced data distributions. Usually, the related research works focus on part of these main challenges or produce black-box models. In this paper, we propose a modified genetic algorithm for simultaneous dimensionality reduction and hyper parameter optimization over imbalanced datasets. The algorithm initialized an eXtreme Gradient Boosting classifier and reduced the features space of tweets dataset; to generate a spam prediction model. The model is validated using a 50 times repeated 10-fold stratified cross-validation, and analyzed using nonparametric statistical tests. The resulted prediction model attains on average 82.32% and 92.67% in terms of geometric mean and accuracy respectively, utilizing less than 10% of the total feature space. The empirical results show that the modified genetic algorithm outperforms Chi 2 and P CA feature selection methods. In addition, eXtreme Gradient Boosting outperforms many machine learning algorithms, including BERT-based deep learning model, in spam prediction. Furthermore, the proposed approach is applied to SMS spam modeling and compared to related works.

Optimal Control Strategy of Construction Machinery Motor Considering Genetic Algorithm

“…In a factory, workers use machines to work, usually for two purposes. The first is to do some auxiliary operations to complete the scheduled tasks, and the second is to enable the robot to achieve the required functions or motion tracks according to requirements, that is, to change the position of the workpiece through operation and change the time to achieve automatic production process [9][10].…”

Modeling the Telemarketing Process Using Genetic Algorithms and Extreme Boosting: Feature Selection and Cost-Sensitive Analytical Approach

“…However, they are still unable to explicitly and clearly explain the basis of a judgment. Genetic Algorithms (GA) have recently been used in simultaneous dimensionality reduction and hyperparameter optimization to tackle challenges imposed by commonly used approaches [22].…”

“…In this research, further modifications are made to the GA presented in [22] (mainly the sensitivity to the class weight and features encoding) to select a feature subset of the marketing dataset and tune the classifier hyperparameters. Cost-Sensitive classification [8], [30] aims at giving more weight to the class of interest (i.e., usually the minority class) and hence minimizing the misclassification cost.…”

“…The work of [22] presented a modified GA that tries to find an optimal subset of dataset features as well as an eXtreme Gradient Boosting (XGBoost) [33] hyperparameters that maximize the fitness function. The utilized fitness function is the Geometric Mean (GMean), which is commonly used in evaluating the prediction performance in imbalanced class distributions.…”

“…The utilized fitness function is the Geometric Mean (GMean), which is commonly used in evaluating the prediction performance in imbalanced class distributions. Equation 4shows the GMean, which is the square root of the multiplication of the positive class recall, i.e., True Positive Rate (TPR), and negative class recall, i.e., True Negative Rate (TNR) [22], [34]- [36]. The TPR and TNR are computed as shown in Equations 6) and 7 respectively.…”