Probing the Hypothesis of SAR Continuity Restoration by the Removal of Activity Cliffs Generators in QSAR

Abstract

In this work we report the first attempt to study the effect of activity cliffs over the generalization ability of machine learning (ML) based QSAR classifiers, using as study case a previously reported diverse and noisy dataset focused on drug induced liver injury (DILI) and more than 40 ML classification algorithms. Here, the hypothesis of structure-activity relationship (SAR) continuity restoration by activity cliffs removal is tested as a potential solution to overcome such limitation. Previously, a parallelism was established between activity cliffs generators (ACGs) and instances that should be misclassified (ISMs), a related concept from the field of machine learning. Based on this concept we comparatively studied the classification performance of multiple machine learning classifiers as well as the consensus classifier derived from predictive classifiers obtained from training sets including or excluding ACGs. The influence of the removal of ACGs from the training set over the virtual screening performance was also studied for the respective consensus classifiers algorithms. In general terms, the removal of the ACGs from the training process slightly decreased the overall accuracy of the ML classifiers and multi-classifiers, improving their sensitivity (the weakest feature of ML classifiers trained with ACGs) but decreasing their specificity. Although these results do not support a positive effect of the removal of ACGs over the classification performance of ML classifiers, the "balancing effect" of ACG removal demonstrated to positively influence the virtual screening performance of multi-classifiers based on valid base ML classifiers. Specially, the early recognition ability was significantly favored after ACGs removal. The results presented and discussed in this work represent the first step towards the application of a remedial solution to the activity cliffs problem in QSAR studies.

Quinolinecarboxamides Inhibit the Replication of the Bovine Viral Diarrhea Virus by Targeting a Hot Spot for the Inhibition of Pestivirus Replication in the RNA-Dependent RNA Polymerase

“…Next, compounds with undetermined IC 50 (>100 µM or <0.02 µM) were removed from the modeling process and bioactivities were transformed to pIC 50 ( pIC 50 = − log 10 IC 50 ). Given the negative effect of the activity cliffs in the QSAR modeling, activity cliff generators (ACGs) were removed from the dataset [35,36]. ACGs were defined as compounds sharing a similarity greater than 65% with any other in the dataset and with pIC 50 differences greater than two in between them.…”

Exploring QSAR models for activity-cliff prediction

“…In the field of computational chemistry, ACs are suspected to form one of the major roadblocks for successful quantitative structure-activity relationship (QSAR) modelling [ 9 , 18 , 37 , 50 ]; abrupt changes in potency are expected to negatively influence machine learning algorithms for pharmacological activity prediction. During the development of QSAR models, ACs are sometimes dismissed as measurement errors [ 39 ], but simply removing ACs from a training data set can result in a loss of precious SAR-information [ 10 ].…”

Exploring QSAR Models for Activity-Cliff Prediction

“…In the field of computational chemistry, ACs are suspected to form one of the major roadblocks for successful quantitative structure-activity relationship (QSAR) modelling [14,26,47,63]; abrupt changes in potency are expected to negatively influence machine learning algorithms for pharmacological activity prediction. During the development of QSAR models, ACs are sometimes dismissed as measurement errors [49], but simply removing ACs from a training data set can result in a loss of precious SAR-information [15].…”