Benchmark of Data Processing Methods and Machine Learning Models for Gut Microbiome-Based Diagnosis of Inflammatory Bowel Disease

Abstract

Patients with inflammatory bowel disease (IBD) wait months and undergo numerous invasive procedures between the initial appearance of symptoms and receiving a diagnosis. In order to reduce time until diagnosis and improve patient wellbeing, machine learning algorithms capable of diagnosing IBD from the gut microbiome’s composition are currently being explored. To date, these models have had limited clinical application due to decreased performance when applied to a new cohort of patient samples. Various methods have been developed to analyze microbiome data which may improve the generalizability of machine learning IBD diagnostic tests. With an abundance of methods, there is a need to benchmark the performance and generalizability of various machine learning pipelines (from data processing to training a machine learning model) for microbiome-based IBD diagnostic tools. We collected fifteen 16S rRNA microbiome datasets (7,707 samples) from North America to benchmark combinations of gut microbiome features, data normalization and transformation methods, batch effect correction methods, and machine learning models. Pipeline generalizability to new cohorts of patients was evaluated with two binary classification metrics following leave-one-dataset-out cross (LODO) validation, where all samples from one study were left out of the training set and tested upon. We demonstrate that taxonomic features processed with a compositional transformation method and batch effect correction with the naive zero-centering method attain the best classification performance. In addition, machine learning models that identify non-linear decision boundaries between labels are more generalizable than those that are linearly constrained. Lastly, we illustrate the importance of generating a curated training dataset to ensure similar performance across patient demographics. These findings will help improve the generalizability of machine learning models as we move towards non-invasive diagnostic and disease management tools for patients with IBD.

The role of gut microbiome in inflammatory bowel disease diagnosis and prognosis

“…Others have selected more bacterial markers as biomarkers for IBD determination using various feature selection methods and ML models. [59][60][61] Manandhar et al 53 Metagenomic sequencing, which has a higher taxonomy resolution that enables better identification of specific bacterial species or strains related to disease development, is increasingly applied for the discovery of microbial markers. Franzosa et al 64 genus data (AUC = 0.869) achieved higher pediatric UC prediction performance than shotgun species data (AUC = 0.763) and pathway data (AUC = 0.764).…”

“…Others have selected more bacterial markers as biomarkers for IBD determination using various feature selection methods and ML models. 59 , 60 , 61 Manandhar et al. 53 selected 50 fecal bacterial taxa for disease diagnosis in a large American cohort.…”

PLSDA-batch: a multivariate framework to correct for batch effects in microbiome data

“…These methods are univariate and were originally developed for gene expression data from microarray or RNA-sequencing. They have been used extensively in microbiome studies [ 30–32 , 46 , 47 ] even though they would require further developments to be adapted to the inherent characteristics of microbiome data. These methods’ limitations include the inability to deal with non-Gaussian distribution, small sample sizes and dependence between microbial variables.…”

“…We assess the performance of PLSDA-batch in extensive simulation studies and three case studies that investigate microbial communities in sponge tissues, anaerobic digestion conditions and diet types in mice. We compare the efficiency of our approaches in removing batch effects and uncovering treatment effects with popular linear methods that have been previously applied in microbial studies [ 30–32 ], such as ComBat and removeBatchEffect. As our approach shares some similarities with Surrogate Variable Analysis (SVA), besides the fact that it accounts, rather than corrects for batch effects, we include some comparisons in the simulation studies.…”

Conditional Forest Models Built Using Metagenomic Data Accurately Predicted Salmonella Contamination in Northeastern Streams

“…Model performance can also be affected by microbiome data preprocessing, such as data normalization ( 28 ). The latter is commonly carried out to account for potential differences in sample library sizes; thus, its effect on prediction accuracy needs to be assessed ( 31 – 33 ).…”