Beyond multidrug resistance: Leveraging rare variants with machine and statistical learning models in Mycobacterium tuberculosis resistance prediction

Abstract

Background The diagnosis of multidrug resistant and extensively drug resistant tuberculosis is a global health priority. Whole genome sequencing of clinical Mycobacterium tuberculosis isolates promises to circumvent the long wait times and limited scope of conventional phenotypic antimicrobial susceptibility, but gaps remain for predicting phenotype accurately from genotypic data especially for certain drugs. Our primary aim was to perform an exploration of statistical learning algorithms and genetic predictor sets using a rich dataset to build a high performing and fast predicting model to detect anti-tuberculosis drug resistance. Methods We collected targeted or whole genome sequencing and conventional drug resistance phenotyping data from 3601 Mycobacterium tuberculosis strains enriched for resistance to first- and second-line drugs, with 1228 multidrug resistant strains. We investigated the utility of (1) rare variants and variants known to be determinants of resistance for at least one drug and (2) machine and statistical learning architectures in predicting phenotypic drug resistance to 10 anti-tuberculosis drugs. Specifically, we investigated multitask and single task wide and deep neural networks, a multilayer perceptron, regularized logistic regression, and random forest classifiers. Findings The highest performing machine and statistical learning methods included both rare variants and those known to be causal of resistance for at least one drug. Both simpler L2 penalized regression and complex machine learning models had high predictive performance. The average AUCs for our highest performing model was 0.979 for first-line drugs and 0.936 for second-line drugs during repeated cross-validation. On an independent validation set, the highest performing model showed average AUCs, sensitivities, and specificities, respectively, of 0.937, 87.9%, and 92.7% for first-line drugs and 0.891, 82.0% and 90.1% for second-line drugs. Our method outperforms existing approaches based on direct association, with increased sum of sensitivity and specificity of 11.7% on first line drugs and 3.2% on second line drugs. Our method has higher predictive performance compared to previously reported machine learning models during cross-validation, with higher AUCs for 8 of 10 drugs. Interpretation Statistical models, especially those that are trained using both frequent and less frequent variants, significantly improve the accuracy of resistance prediction and hold promise in bringing sequencing technologies closer to the bedside.

Improved inference and prediction of bacterial genotype-phenotype associations using pangenome-spanning regressions

“…Using simulated data we demonstrate improved power and false-discovery rate at the single variant level compared with fixed and random effect models, and illustrate this use in practise on antibiotic resistance phenotypes in two species. We show further results which find similar accuracy between new machine-learning and simpler approaches, consistent with previous studies (4,5,24) . Additionally, our approach was able to estimate trait heritability without assuming specific effect size distributions, which are unproven in bacterial populations.…”

“…Previous work has evaluated the use of a multitask deep neural network, and when comparing this to lasso regression found comparable accuracy (5) . Using the same input of ~6500 short variants across the allele-frequency spectrum for these 3566 samples (split into training and test datasets) led to an average false-negative rate of 2% ± 3% in the unweighted model 3% ± 4% in the weighted model, and false-positive rate of 11% ± 8% in the unweighted model, 12% ± 10% in the weighted model (Supplementary table 5).…”

“…Sequence assemblies were available from the original publications, with the exception of one N. gonorrhoeae study (47) . For this study we downloaded the read data, removed adapter sequences with trimmomatic (48) (5) . For the Massachusetts and Maela datasets, we used SNP calls from an earlier GWAS in this population (50) .…”

“…With enough independent observations, these methods can automatically predict the phenotype of new isolates, and potentially tell us something about the underlying genetic mechanisms. A deluge of recent papers have applied general predictive models to such datasets, most showing high accuracy (3)(4)(5)(6)(7)(8) , though some commentaries have been more cautious in their conclusions (9,10) .…”

“…They are potentially broadly applicable to any problem with vast amounts of data, though perform best when the number of datapoints exceeds the number of dimensions. Unsurprisingly their uptake in sequence analysis (17,18) and bacterial genomics specifically has been rapid (4,5) .…”