A Novel Automated Framework for QSAR Modeling of Highly Imbalanced Leishmania High-Throughput Screening Data

Abstract

In silico prediction of antileishmanial activity using quantitative structure−activity relationship (QSAR) models has been developed on limited and small datasets. Nowadays, the availability of large and diverse high-throughput screening data provides an opportunity to the scientific community to model this activity from the chemical structure. In this study, we present the first KNIME automated workflow to modeling a large, diverse, and highly imbalanced dataset of compounds with antileishmanial activity. Because the data is strongly biased toward inactive compounds, a novel strategy was implemented based on the selection of different balanced training sets and a further consensus model using single decision trees as the base model and three criteria for output combinations. The decision tree consensus was adopted after comparing its classification performance to consensuses built upon Gaussian-Naıve-Bayes, Support-Vector-Machine, Random-Forest, Gradient-Boost, and Multi-Layer-Perceptron base models. All these consensuses were rigorously validated using internal and external test validation sets and were compared against each other using Friedman and Bonferroni−Dunn statistics. For the retained decision tree-based consensus model, which covers 100% of the chemical space of the dataset and with the lowest consensus level, the overall accuracy statistics for test and external sets were between 71 and 74% and 71 and 76%, respectively, while for a reduced chemical space (21%) and with an incremental consensus level, the accuracy statistics were substantially improved with values for the test and external sets between 86 and 92% and 88 and 92%, respectively. These results highlight the relevance of the consensus model to prioritize a relatively small set of active compounds with high prediction sensitivity using the Incremental Consensus at high level values or to predict as many compounds as possible, lowering the level of Incremental Consensus. Finally, the workflow developed eliminates human bias, improves the procedure reproducibility, and allows other researchers to reproduce our design and use it in their own QSAR problems.

Isometric Stratified Ensembles: A Partial and Incremental Adaptive Applicability Domain and Consensus-Based Classification Strategy for Highly Imbalanced Data Sets with Application to Colloidal Aggregation

“…To determine the most relevant variables for the classification of SCAMs based on a DT strategy, the same variable selection algorithm as the one published in the paper by Casanova-Alvarez et al . was implemented: a selection of variables by permutation using a decision tree algorithm combined with a recursive selection of least correlated variables. This algorithm is encapsulated in a component named “variable selection by decision tree”.…”

“…Strata can be gathered based on both the partial level and incremental level of statistic performance. Our strategy is an extension from 1D to 2D stratification of work recently published by Casanova-Alvarez and coauthors and an evolution in terms of classification and recursive variable selection of work published by Sheridan. , ISE is a machine learning model agnostic and thus can be used in combination to any classification base model as illustrated in this study. ISE predictive models are implemented using the KNIME open-source software, and their workflows for prediction with full data are available in open-access…”

Tuning gradient boosting for imbalanced bioassay modelling with custom loss functions

“…While there are several studies investigating resampling in the context of bioassay modelling [ 5 , 28 – 30 ], changing the training objective has not been thoroughly investigated thus far. This study directly addresses this gap by investigating the effectiveness of a variety of recently published imbalance-insensitive loss functions for training Gradient Boosting classifiers.…”

The (Re)-Evolution of Quantitative Structure–Activity Relationship (QSAR) Studies Propelled by the Surge of Machine Learning Methods

“…In another contribution, an automated workflow was created to build a classification-based model for diverse and imbalanced data sets. 38 This workflow was tested using a data set composed of 196 173 compounds, with 1063 compounds displaying antileishmanial activity. Six different methods were tested to build a consensus model, and the model using decision trees had the best performance.…”

“…DLCA showed a better performance for the two data sets, compared to other consensus approaches. In another contribution, an automated workflow was created to build a classification-based model for diverse and imbalanced data sets . This workflow was tested using a data set composed of 196 173 compounds, with 1063 compounds displaying antileishmanial activity.…”