Using Feature Selection with Bagging and Rule Extraction in Drug Discovery

Abstract

Abstract. This paper investigates different ways of combining feature selection with bagging and rule extraction in predictive modeling. Experiments on a large number of data sets from the medicinal chemistry domain, using standard algorithms implemented in the Weka data mining workbench, show that feature selection can lead to significantly improved predictive performance. When combining feature selection with bagging, employing the feature selection on each bootstrap obtains the best result. When using decision trees for rule extraction, the effect of feature selection can actually be detrimental, unless the transductive approach oracle coaching is also used. However, employing oracle coaching will lead to significantly improved performance, and the best results are obtained when performing feature selection before training the opaque model. The overall conclusion is that it can make a substantial difference for the predictive performance exactly how feature selection is used in conjunction with other techniques.

Impact of freeway weaving segment design on light-duty vehicle exhaust emissions

“…BDT is an embedded method to select actual feature by the decision tree learning algorithm during model construction. Typically, it is choosing different split attributes by a decision tree algorithm (Johansson et al, 2010). Decision trees on a data set are bagged, a number of bootstrap replicas of the data set are generated, and decision trees on these replicas grow.…”