Purpose: Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis. Methods: A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance. Results: After screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance. Conclusion: This systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.

Dosing of vancomycin is often guided by therapeutic drug monitoring and population pharmacokinetic models in the intensive care unit (ICU). The validity of these models is crucial, as ICU patients have marked pharmacokinetic variability. Therefore, we set out to evaluate the predictive performance of published population pharmacokinetic models of vancomycin in ICU patients. The PubMed database was used to search for population pharmacokinetic models of vancomycin in adult ICU patients. The identified models were evaluated in two independent data sets which were collected from two large hospitals in the Netherlands (Amsterdam UMC, Location VUmc, and OLVG Oost). We also tested a one-compartment model with fixed values for clearance and volume of distribution, in which a clinical standard dosage regimen (SDR) was mimicked to assess its predictive performance. Prediction error was calculated to assess the predictive performance of the models. Six models plus the SDR model were evaluated. The model of Roberts et al. (J. A. Roberts, F. S. Taccone, A. A. Udy, J.-L. Vincent, F. Jacobs, and J. Lipman, Antimicrob Agents Chemother 55:2704–2709, 2011, https://doi.org/10.1128/AAC.01708-10) performed satisfactorily, with mean and median values of prediction error of 5.1% and −7.5%, respectively, for Amsterdam UMC, Location VUmc, patients, and −12.6% and −17.2% respectively, for OLVG Oost patients. The other models, including the SDR model, yielded high mean values (−49.7% to 87.7%) and median values (−56.1% to 66.1%) for both populations. In conclusion, only the model of Roberts et al. was able to validly predict the concentrations of vancomycin for our data, whereas other models and standard dosing were largely inadequate. Extensive evaluation should precede the adoption of any model in clinical practice for ICU patients.