A self-adaptive level-based learning artificial bee colony algorithm for feature selection on high-dimensional classification

Abstract

Feature selection is an important data preprocessing method in data mining and machine learning, yet it faces the challenge of "curse of dimensionality" when dealing with high-dimensional data. In this paper, a self-adaptive level-based learning artificial bee colony (SLLABC) algorithm is proposed for high-dimensional feature selection problem. The SLLABC algorithm includes three new mechanisms: (1) A novel level-based learning mechanism is introduced to accelerate the convergence of the basic artificial bee colony algorithm, which divides the population into several levels and the individuals on each level learn from the individuals on higher levels, especially, the individuals on the highest level learn from each other. (2) A self-adaptive method is proposed to keep the balance between exploration and exploitation abilities, which takes the diversity of population into account to determine the number of levels. The lower the diversity is, the fewer the levels are divided. (3) A new update mechanism is proposed to reduce the number of selected features. In this mechanism, if the error rate of an offspring is higher than or is equal to that of its parent but selects more features, then the offspring is discarded and the parent is retained, otherwise, the offspring replaces its parent. Further, we discuss and analyze the contribution of these novelties to the diversity of population and the performance of classification. Finally, the results, compared with 8 state-of-the-art algorithms on 12 high-dimensional datasets, confirm the competitive performance of the proposed SLLABC on both classification accuracy and the size of the feature subset.

Feature Selection Problem and Metaheuristics: A Systematic Literature Review about Its Formulation, Evaluation and Applications

“…(b) Error rate: Defined in detail in Section 4.2.1 and mathematically in equation 26. This objective function has been pursued in [42][43][44][45][46][47][48][49][50][51].…”

“…Measures the overall accuracy of a model in terms of the proportion of misclassified instances in a dataset. This metric has been used in [46,49,59,62,[64][65][66][67][68]70,71,93,120,122,131,133,134,140,150,179] and mathematically is defined as follows:…”

“…The authors report the average number of features selected in the performed runs and their standard deviation. This metric has been used in [21,27,28,32,34,[36][37][38]40,42,[44][45][46][47][48][49][50][51][53][54][55]57,58,[63][64][65][66][67][68][69][70][71][72]75,77,78,[81][82][83][84][85][86][87][88][89][90][91][92][93][94][95]…”

“…From the data presented in Figure 32, a predominant trend emerges related to the utilization of benchmark datasets [27][28][29][30][31][32][33][34][36][37][38][39][40][41][42][43][46][47][48][49][50][53][54][55][56][57][58][59][60][62][63][64][65][66][67][68][69][70][71][72][73][74][75][77][78][79][80][81][82][142]…”