Random forests for feature selection in QSPR Models - an application for predicting standard enthalpy of formation of hydrocarbons

Abstract

BackgroundOne of the main topics in the development of quantitative structure-property relationship (QSPR) predictive models is the identification of the subset of variables that represent the structure of a molecule and which are predictors for a given property. There are several automated feature selection methods, ranging from backward, forward or stepwise procedures, to further elaborated methodologies such as evolutionary programming. The problem lies in selecting the minimum subset of descriptors that can predict a certain property with a good performance, computationally efficient and in a more robust way, since the presence of irrelevant or redundant features can cause poor generalization capacity. In this paper an alternative selection method, based on Random Forests to determine the variable importance is proposed in the context of QSPR regression problems, with an application to a manually curated dataset for predicting standard enthalpy of formation. The subsequent predictive models are trained with support vector machines introducing the variables sequentially from a ranked list based on the variable importance.ResultsThe model generalizes well even with a high dimensional dataset and in the presence of highly correlated variables. The feature selection step was shown to yield lower prediction errors with RMSE values 23% lower than without feature selection, albeit using only 6% of the total number of variables (89 from the original 1485). The proposed approach further compared favourably with other feature selection methods and dimension reduction of the feature space. The predictive model was selected using a 10-fold cross validation procedure and, after selection, it was validated with an independent set to assess its performance when applied to new data and the results were similar to the ones obtained for the training set, supporting the robustness of the proposed approach.ConclusionsThe proposed methodology seemingly improves the prediction performance of standard enthalpy of formation of hydrocarbons using a limited set of molecular descriptors, providing faster and more cost-effective calculation of descriptors by reducing their numbers, and providing a better understanding of the underlying relationship between the molecular structure represented by descriptors and the property of interest.

Investigation of the influence of protein corona composition on gold nanoparticle bioactivity using machine learning approaches

“…The method comes therefore to a non-linear consensus on unpruned decision trees. Two important variables (although the method is not very sensitive to their values) are the number of trees to grow in the forest (ntree) and the number of variables to choose at each node (mtry) [60,62]. Using RF there is a reduced risk of overfitting, since this approach uses a large number of simple models and includes the possibility to treat non-standard problems (number of descriptors higher than that of observations).…”

Formation enthalpies for transition metal alloys using machine learning

“…A popular approach in the literature is to apply tools from machine learning on certain DFT calculations to accelerate prediction of various properties of compounds [16][17][18][19][20][21][22] . Ideas from machine learning have been coupled with databases of ab initio calculations to estimate molecular electronic properties in chemical compound space, including the enthalpy of formation of compounds 23,24 . However, these methods still have the major disadvantage of requiring results from many DFT calculations, which may not be possible for alloys without given crystal structures, i.e., amorphous or noncrystalline alloys.…”

Predicting Subtype Selectivity for Adenosine Receptor Ligands with Three-Dimensional Biologically Relevant Spectrum (BRS-3D)

“…Random forest (RF) was used for feature selection. RF was a popular and efficient algorithm, based on model aggregation ideas, regardless of classification or regression problems37. RF was implemented by the component “Learn R Forest Model” in Pipeline Pilot 8.5, invoking the R package “RandomForest”.…”