Fusing Dual-Event Data Sets for Mycobacterium tuberculosis Machine Learning Models and Their Evaluation

Abstract

The search for new tuberculosis treatments continues as we need to find molecules that can act more quickly, be accommodated in multi-drug regimens, and overcome ever increasing levels of drug resistance. Multiple large scale phenotypic high-throughput screens against Mycobacterium tuberculosis (Mtb) have generated dose response data, enabling the generation of machine learning models. These models also incorporated cytotoxicity data and were recently validated with a large external dataset. A cheminformatics data-fusion approach followed by Bayesian machine learning, Support Vector Machine or Recursive Partitioning model development (based on publicly available Mtb screening data) was used to compare individual datasets and subsequent combined models. A set of 1924 commercially available molecules with promising antitubercular activity (and lack of relative cytotoxicity to Vero cells) were used to evaluate the predictive nature of the models. We demonstrate that combining three datasets incorporating antitubercular and cytotoxicity data in Vero cells from our previous screens results in external validation receiver operator curve (ROC) of 0.83 (Bayesian or RP Forest). Models that do not have the highest five-fold cross validation ROC scores can outperform other models in a test set dependent manner. We demonstrate with predictions for a recently published set of Mtb leads from GlaxoSmithKline that no single machine learning model may be enough to identify compounds of interest. Dataset fusion represents a further useful strategy for machine learning construction as illustrated with Mtb. Coverage of chemistry and Mtb target spaces may also be limiting factors for the whole-cell screening data generated to date.

Are Bigger Data Sets Better for Machine Learning? Fusing Single-Point and Dual-Event Dose Response Data for Mycobacterium tuberculosis

“…Thus, addition of the single-point inactives to the dose response inactives should significantly enhance a model’s knowledge of antitubercular “inactivity.” The largest combined model we can create from these datasets has 345,011 molecules. These new models are enhanced with regard to their number of inactives (Table 1) and their coverage of chemical property space as assessed using PCA plots generated with all training data and the ARRA compounds or GSK actives (Figure 2), compared with a similar plot generated earlier with all dose-response compounds 22 .…”

“…Of course these approaches could be repeated with different machine learning algorithms although we have shown little effect across models for this data previously. 22 …”

“… a Where: IC 90 < 10 μg/ml (TAACF-CB2 only) or 10 μM (other datasets) and a selectivity index (SI) greater than ten where the SI is calculated from SI = CC 50 /IC 90 . b Previously published 22 …”

“…22 We reported the examination of Support Vector Machine (SVM) and Recursive Partitioning (RP) single tree and forest models to compare with dual-event Bayesian models of in vitro antitubercular efficacy and acceptable Vero cell cytotoxicity (selectivity index SI = (MIC or IC 90 )/CC 50 ≥ 10; where MIC = minimum compound concentration to inhibit growth of organism usually by 90 or 99%, IC 90 = compound concentration necessary to inhibit 90% of the organism’s growth, CC 50 = compound concentration that inhibits growth of the cells by 50%). 22 We did not find a dramatic difference between the Bayesian and other models for the same individual datasets when performing 5-fold cross validation. The ability of a model to predict hits amongst the GlaxoSmithKline (GSK) set of 177 antituberculars 26 , in fact, appeared to depend more on the identity of the training set than on the method used.…”

Collaborative drug discovery for More Medicines for Tuberculosis (MM4TB)

“…Fusion of three dual activity models gave an excellent ROC value with a fourth external dataset from the same laboratory [61]. These models have also been used individually with a testset of 1924 molecules for which cytotoxicity was determined in three cell lines and enrichments of 11.8-fold were observed in the best case [36].…”