Determining threshold value on information gain feature selection to increase speed and prediction accuracy of random forest

Abstract

Feature selection is a pre-processing technique used to remove unnecessary characteristics, and speed up the algorithm's work process. A part of the technique is carried out by calculating the information gain value of each dataset characteristic. Also, the determined threshold rate from the information gain value is used in feature selection. However, the threshold value is used freely or through a rate of 0.05. Therefore this study proposed the threshold rate determination using the information gain value’s standard deviation generated by each feature in the dataset. The threshold value determination was tested on 10 original datasets transformed by FFT and IFFT and classified using Random Forest. On processing the transformed dataset with the proposed threshold this study resulted in lower accuracy and longer execution time compared to the same process with Correlation-Base Feature Selection (CBF) and a standard 0.05 threshold method. Similarly, the required accuracy value is lower when using transformed features. The study showed that by processing the original dataset with a standard deviation threshold resulted in better feature selection accuracy of Random Forest classification. Furthermore, by using the transformed feature with the proposed threshold excluding the imaginary numbers leads to a faster average time than the three methods compared.

The accuracy of Random Forest performance can be improved by conducting a feature selection with a balancing strategy

“…This study followed previous studies ( Prasetiyowati, Maulidevi & Surendro, 2021 ; Prasetiyowati, Maulidevi & Surendro, 2020a ; Prasetiyowati, Maulidevi & Surendro, 2020b ). The researchers began this study by using the Correlation-based Feature Selection (CBF) for feature selection.…”

“…Random Forest is a classification algorithm based on the random selection of trees ( Gounaridis & Koukoulas, 2016 ; Prasetiyowati, Maulidevi & Surendro, 2020a ; Prasetiyowati, Maulidevi & Surendro, 2021 ), thereby making it uninformative as a tool used to build the decision tree ( Breiman, 2001 ; Prasetiyowati, Maulidevi & Surendro, 2021 ; Scornet, Biau & Vert, 2015 ). However, this process allows the selected feature to be uninformative.…”

“…Therefore, improving the feature selection process is necessary to make it informative with a faster execution time. Several studies have proposed the feature selection process for Random Forest ( Adnan, 2014 ; Prasetiyowati, Maulidevi & Surendro, 2021 ; Sun et al, 2020 ; Ye et al, 2013 ; Zhang & Suganthan, 2014 ), including the use of IG with a threshold based on the standard deviation value ( Prasetiyowati, Maulidevi & Surendro, 2021 ). Zhang & Suganthan (2014) proposed a new method in Random Forest by increasing tree diversity by combining a different rotation space at the root node.…”

“…Several other studies use the calculation of the frequency of each feature to determine the threshold value as a subset of the final features ( Tsai & Sung, 2020 ). However, some also use the standard deviation to determine the threshold ( Prasetiyowati, Maulidevi & Surendro, 2021 ; Sindhu & Radha, 2020 ).…”

Objective Phenotyping of Root System Architecture Using Image Augmentation and Machine Learning in Alfalfa (Medicago sativa L.)

“…ML can select significant predictors and exclude collinear variables, whereas unsupervised ML uses all the predictors with the same weights. Weights of RSA traits affected ML models in numerous other studies [ 80 – 83 ]. In this study, we segmented root crowns and used RhizoVision Explorer to extract root traits for use in these models.…”