Prediction performance of the machine learning model in predicting mortality risk in patients with traumatic brain injuries: a systematic review and meta-analysis

Abstract

Purpose With the in-depth application of machine learning(ML) in clinical practice, it has been used to predict the mortality risk in patients with traumatic brain injuries(TBI). However, there are disputes over its predictive accuracy. Therefore, we implemented this systematic review and meta-analysis, to explore the predictive value of ML for TBI. Methodology We systematically retrieved literature published in PubMed, Embase.com, Cochrane, and Web of Science as of November 27, 2022. The prediction model risk of bias(ROB) assessment tool (PROBAST) was used to assess the ROB of models and the applicability of reviewed questions. The random-effects model was adopted for the meta-analysis of the C-index and accuracy of ML models, and a bivariate mixed-effects model for the meta-analysis of the sensitivity and specificity. Result A total of 47 papers were eligible, including 156 model, with 122 newly developed ML models and 34 clinically recommended mature tools. There were 98 ML models predicting the in-hospital mortality in patients with TBI; the pooled C-index, sensitivity, and specificity were 0.86 (95% CI: 0.84, 0.87), 0.79 (95% CI: 0.75, 0.82), and 0.89 (95% CI: 0.86, 0.92), respectively. There were 24 ML models predicting the out-of-hospital mortality; the pooled C-index, sensitivity, and specificity were 0.83 (95% CI: 0.81, 0.85), 0.74 (95% CI: 0.67, 0.81), and 0.75 (95% CI: 0.66, 0.82), respectively. According to multivariate analysis, GCS score, age, CT classification, pupil size/light reflex, glucose, and systolic blood pressure (SBP) exerted the greatest impact on the model performance. Conclusion According to the systematic review and meta-analysis, ML models are relatively accurate in predicting the mortality of TBI. A single model often outperforms traditional scoring tools, but the pooled accuracy of models is close to that of traditional scoring tools. The key factors related to model performance include the accepted clinical variables of TBI and the use of CT imaging.

Artificial Intelligence and Machine Learning Applications in Critically Ill Brain Injured Patients

“…Established multivariate models, such as the Corticosteroid Randomization After Significant Head Injury (CRASH) and International Mission on Prognosis and Analysis of Clinical Trials in TBI (IMPACT), are built on trial data and incorporate a mix of demographic, clinical, and radiographic information to predict mortality and neurological outcomes. 40 Although these tools provide a reasonable estimation of injury severity and correlate with outcomes, they are limited by wide confidence intervals and historical data, which may hinder their applicability in contemporary clinical settings. 41 ML algorithms have demonstrated significant potential in refining these predictions by incorporating newer clinical data, often requiring smaller patient cohorts.…”

“…42,43 No single ML algorithm has emerged as universally superior as different supervised learning techniques may be better suited to specific types of data and inputs. 40,46 Despite these variations, recent systematic reviews have largely affirmed the robust predictive capacity of ML models in TBI prognosis, typically outperforming traditional regression models, and show a trend of improved performance in line with technological advancements in ML. 40,42 ML-based models have not only validated the prognostic relevance of established clinical features such as age, Glasgow Coma Scale (GCS) scores, and imaging findings, but have also highlighted the predictive significance of serum metabolites that were not included in conventional models, underscoring the role of multi-organ dysfunction in TBI pathophysiology.…”

“…40,46 Despite these variations, recent systematic reviews have largely affirmed the robust predictive capacity of ML models in TBI prognosis, typically outperforming traditional regression models, and show a trend of improved performance in line with technological advancements in ML. 40,42 ML-based models have not only validated the prognostic relevance of established clinical features such as age, Glasgow Coma Scale (GCS) scores, and imaging findings, but have also highlighted the predictive significance of serum metabolites that were not included in conventional models, underscoring the role of multi-organ dysfunction in TBI pathophysiology. 42,46 Another advantage of ML models is their capacity to integrate dynamic, high-resolution data in real-time, such as vital signs and intracranial measurements, offering more precise and timely prognostic insights compared to conventional models using static data collected at admission or aggregated daily averages.…”