Machine learning performance in a microbial molecular autopsy context: A cross-sectional postmortem human population study

Abstract

Background The postmortem microbiome can provide valuable information to a death investigation and to the human health of the once living. Microbiome sequencing produces, in general, large multi-dimensional datasets that can be difficult to analyze and interpret. Machine learning methods can be useful in overcoming this analytical challenge. However, different methods employ distinct strategies to handle complex datasets. It is unclear whether one method is more appropriate than others for modeling postmortem microbiomes and their ability to predict attributes of interest in death investigations, which require understanding of how the microbial communities change after death and may represent those of the once living host. Methods and findings Postmortem microbiomes were collected by swabbing five anatomical areas during routine death investigation, sequenced and analyzed from 188 death cases. Three machine learning methods (boosted algorithms, random forests, and neural networks) were compared with respect to their abilities to predict case attributes: postmortem interval (PMI), location of death, and manner of death. Accuracy depended on the method used, the numbers of anatomical areas analyzed, and the predicted attribute of death. Conclusions All algorithms performed well but with distinct features to their performance. Xgboost often produced the most accurate predictions but may also be more prone to overfitting. Random forest was the most stable across predictions that included more anatomic areas. Analysis of postmortem microbiota from more than three anatomic areas appears to yield limited returns on accuracy, with the eyes and rectum providing the most useful information correlating with circumstances of death in most cases for this dataset.

Evaluating Bioinformatic Pipeline Performance for Forensic Microbiome Analysis*,†,‡

“…This study included comparison of bioinformatic pipelines using machine learning outcomes. Rather than an exhaustive search of machine learning algorithms , we used a standardized user‐friendly methodology of random forest classification (out‐of‐bag error) after comparison to the test‐set validations were within 2% error rate as OOB error rate. QIIME2 had a higher overall classification error than mothur, but resulted in <1% difference from mothur at the family taxonomic level.…”

“…However, test‐set validation (70% training sets, 30% test sets) was also tested and the error rates were within 2% of the OOB error rates. For a more extensive comparison of random forest methods on the larger dataset of these postmortem microbiome data, please see Zhang et al .…”

“…Downstream analytical methods for forensic microbiology studies are still being evaluated. Recently, machine learning algorithms and parameterization were comprehensively tested for potential direct forensic applications to postmortem interval estimation, manner‐of‐death determination, and location of death (i.e., inside, outside, hospital death) for this particular dataset . Therefore, we focus on preceding steps of microbial analysis before modeling applications for comparing forensic predictions.…”

Forensic Prediction using Bias-Variance Tradeoff and Random Forest Algorithms

“…The microbial behavior observed within and after 48 hours of death for 188 death cases [1] are given as training data into the machine learning model. Data with low variance is processed by using Random forest method.…”

“…Hence, Bias-Variance Tradeoff is necessary. Random forest algorithm is used to find the prediction and the sample prediction with five anatomical areas from three combinations of predictor variables are given in the table I, II and III below [1]. In the tables described above, we observe that the result is not at its best predictive accuracy.…”