All-Assay-Max2 pQSAR: Activity Predictions as Accurate as Four-Concentration IC₅₀s for 8558 Novartis Assays

Abstract

Profile-QSAR (pQSAR) is a massively multi-task, 2-step machine learning method with unprecedented scope, accuracy and applicability domain. In step one, a "profile" of conventional single-assay random forest regression (RFR) models are trained on a very large number of biochemical and cellular pIC50 assays using Morgan 2 sub-structural fingerprints as compound descriptors. In step two, a panel of PLS models are built using the profile of pIC50 predictions from those RFR models as compound descriptors. Hence the name. Previously described for a panel of 728 biochemical and cellular kinase assays, we have now built an enormous pQSAR from 11,805 diverse Novartis IC50 and EC50 assays. This large number of assays, and hence of compound descriptors for PLS, dictated reducing the profile by only including RFR models whose predictions correlate with the assay being modeled. The RFR and pQSAR models were evaluated with our "realistically novel" held-out test set whose median average similarity to the nearest training set member across the 11,805 assays was only 0.34, thus testing a realistically large applicability domain. For the 11,805 single-assay RFR models, the median correlation of prediction with experiment was only R 2 ext=0.05, virtually random, and only 8% of the models achieved our standard success threshold of R 2 ext=0.30. For pQSAR, the median correlation was R 2 ext=0.53, comparable to 4-concentration experimental IC50s, and 72% of the models met our R 2 ext>0.30 standard, totaling 8558 successful models. The successful models included assays from all of the 51 annotated target sub-classes, as well as 4196 phenotypic assays, indicating that pQSAR can be applied to virtually any disease area. Every month, all models are updated to include new measurements, and predictions are made for 5.5 million Novartis compounds, totaling 50 billion predictions. Common uses have included virtual screening, selectivity design, toxicity and promiscuity prediction, mechanism-of-action prediction, and others.

Extension of pQSAR: Ensemble Model Generated by Random Forest and Partial Least Squares Regressions

“…Since R 2 = 0.3 is suggested as standard success threshold in [29], we evaluate the performance of the model with the number of kinases with an average of R 2 values ≥ 0.3. We summarize the number of kinases with an average R 2 value Tables 10 and 11 for the performances done above.…”

“…In order to check the goodness of prediction for a QASR model Ψ, using a test set {(x j , z j )} t j=1 , we use the Pear- [27]- [29].…”

“…We compare the performance with RFRs applied for the initialization as given in (2) as well as the pQSAR model suggested by Martin et. al [27]- [29] as a baseline model because the motive that led us to build the proposed model was inspired by the pSQAR model and began with an attempt to improve the performance of this model. Also, the R 2 value is calculated for each iteration in order to analyze the degree of improvement for iteration, which can be used to determine the optimal number of iterations.…”

“…In the data table, 94.5% of the values (1,983,209 out of 159 assays × 13,192 compounds) are missing. For each assay, we split the compounds with a value into 25% for testing and 75% for training sets, using the python program in the supporting information of [29] to generate the realistic training/test set splits. We convert the IC 50 bio-activity values into pIC 50 values.…”

“…Among these attempts, Martin et. al [27]- [29] introduced a distinguishable predictive ensemble method using bioactivity assay data with missing values, named profile QSAR 2.0 (pQSAR). The profile of a chemical compound for a set of assays is just the corresponding bio-activity values in the assays.…”