The derivation of an International Classification of Diseases, Tenth Revision–based trauma-related mortality model using machine learning

Tran, Zachary; Zhang, Wenhao; Verma, Arjun; Cook, Alan; Kim, Dennis; Burruss, Sigrid; Ramezani, Ramin; Benharash, Peyman

doi:10.1097/ta.0000000000003416

Cited by 10 publications

(15 citation statements)

References 27 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In agreement with our prior work, ML-based models were shown to have improved performance compared to preexisting injury tools [ 13 ]. These findings were anticipated given the XGBoost model’s greater ROC and better calibration following injury variable-only adjustment compared to ISS and TMPM.…”

Section: Discussionsupporting

confidence: 87%

“…They contain descriptors for “initial encounter”, “subsequent encounter”, and “sequela.” To ensure that only first-time injuries were evaluated, analysis was limited to injury codes that specify “initial encounter.” Codes are compiled at the end of each patient’s hospitalization using documentation from medical examiners and operative reports, radiologic studies as well as physicians’ notes. In the present study, 8,021 ICD-10-CM codes were grouped by clinical relevance into 1,495 final variables, as previously described by our group [ 13 ]. Notably, both ISS and ICD-10-CM nomenclature describe “unsurvivable” injuries.…”

Section: Methodsmentioning

confidence: 99%

“…To assess generalizability across patient cohorts, sensitivity analysis was performed on six subgroups of patients, including those (1) with head injuries, (2) without head injuries, (3) with penetrating or (4) blunt traumatic mechanisms, (5) <50 years old and (6) ≥50 years old. Head injuries were defined as patients who had at least one cranial injury code as previously defined [ 13 ].…”

Section: Methodsmentioning

confidence: 99%

“…Its prior applications have included predicting complications following shoulder arthroplasty, bleeding following colonic resection, among others [ 12 ]. In fact, recent work from our group demonstrated improved discrimination and calibration of eXtreme gradient boosting (XGBoost), a ML approach, in mortality prediction compared to logistic regression, ISS and TMPM [ 13 ].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ICD-10 based machine learning models outperform the Trauma and Injury Severity Score (TRISS) in survival prediction

et al. 2022

View full text Add to dashboard Cite

Background Precise models are necessary to estimate mortality risk following traumatic injury to inform clinical decision making or quantify hospital performance. The Trauma and Injury Severity Score (TRISS) has been the historical gold standard in survival prediction but its limitations are well-characterized. The present study used International Classification of Diseases 10th Revision (ICD-10) injury codes with machine learning approaches to develop models whose performance was compared to that of TRISS. Methods The 2015–2017 National Trauma Data Bank was used to identify patients following trauma-related admission. Injury codes from ICD-10 were grouped by clinical relevance into 1,495 variables. The TRISS score, which comprises the Injury Severity Score, age, mechanism (blunt vs penetrating) as well as highest 24-hour values for systolic blood pressure (SBP), respiratory rate (RR) and Glasgow Coma Scale (GCS) was calculated for each patient. A base eXtreme gradient boosting model (XGBoost), a machine learning technique, was developed using injury variables as well as age, SBP, RR, mechanism and GCS. Prediction of in-hospital survival and other in-hospital complications were compared between both models using receiver operating characteristic (ROC) and reliability plots. A complete XGBoost model, containing injury variables, vitals, demographic information and comorbidities, was additionally developed. Results Of 1,380,740 patients, 1,338,417 (96.9%) survived to discharge. Compared to survivors, those who died were older and had a greater prevalence of penetrating injuries (18.0% vs 9.44%). The base XGBoost model demonstrated a greater receiver-operating characteristic (ROC) than TRISS (0.950 vs 0.907) which persisted across sub-populations and secondary endpoints. Furthermore, it exhibited high calibration across all risk levels (R2 = 0.998 vs 0.816). The complete XGBoost model had an exceptional ROC of 0.960. Conclusions We report improved performance of machine learning models over TRISS. Our model may improve stratification of injury severity in clinical and quality improvement settings.

show abstract

Section: Discussionsupporting

confidence: 87%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ICD-10 based machine learning models outperform the Trauma and Injury Severity Score (TRISS) in survival prediction

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The xgboost and sklearn packages of Python were used to develop predictive models as described above [ 23 , 27 ]. The Python code to develop and evaluate our ML models has been previously published by our group [ 28 ].…”

Section: Methodsmentioning

confidence: 99%

Machine learning-based modeling of acute respiratory failure following emergency general surgery operations

et al. 2022

View full text Add to dashboard Cite

Background Emergency general surgery (EGS) operations are associated with substantial risk of morbidity including postoperative respiratory failure (PRF). While existing risk models are not widely utilized and rely on traditional statistical methods, application of machine learning (ML) in prediction of PRF following EGS remains unexplored. Objective The present study aimed to develop ML-based prediction models for respiratory failure following EGS and compare their performance to traditional regression models using a nationally-representative cohort. Methods Non-elective hospitalizations for EGS (appendectomy, cholecystectomy, repair of perforated ulcer, large or small bowel resection, lysis of adhesions) were identified in the 2016–18 Nationwide Readmissions Database. Factors associated with PRF were identified using ML techniques and logistic regression. The performance of XGBoost and logistic regression was evaluated using the receiver operating characteristic curve and coefficient of determination (R2). The impact of PRF on mortality, length of stay (LOS) and hospitalization costs was secondarily assessed using generalized linear models. Results Of 1,003,703 hospitalizations, 8.8% developed PRF. The XGBoost model exhibited slightly superior discrimination compared to logistic regression (0.900, 95% CI 0.899–0.901 vs 0.894, 95% CI 0.862–0.896). Compared to logistic regression, XGBoost demonstrated excellent calibration across all risk levels (R2: 0.998 vs 0.962). Congestive heart failure, neurologic disorders, and coagulopathy were significantly associated with increased risk of PRF. After risk-adjustment, PRF was associated with 10-fold greater odds (95% confidence interval (CI) 9.8–11.1) of mortality and incremental increases in LOS by 3.1 days (95% CI 3.0–3.2) and $11,900 (95% CI 11,600–12,300) in costs. Conclusions Logistic regression and XGBoost perform similarly in overall classification of PRF risk. However, due to superior calibration at extremes of risk, ML-based models may prove more useful in the clinical setting, where probabilities rather than classifications are desired.

show abstract

Development and Validation of a Model to Quantify Injury Severity in Real Time

Choi,

Vendrow,

Moor

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceQuantifying injury severity is integral to trauma care benchmarking, decision-making, and research, yet the most prevalent metric to quantify injury severity—Injury Severity Score (ISS)— is impractical to use in real time.ObjectiveTo develop and validate a practical model that uses a limited number of injury patterns to quantify injury severity in real time through 3 intuitive outcomes.Design, Setting, and ParticipantsIn this cohort study for prediction model development and validation, training, development, and internal validation cohorts comprised 223 545, 74 514, and 74 514 admission encounters, respectively, of adults (age ≥18 years) with a primary diagnosis of traumatic injury hospitalized more than 2 days (2017-2018 National Inpatient Sample). The external validation cohort comprised 3855 adults admitted to a level I trauma center who met criteria for the 2 highest of the institution’s 3 trauma activation levels.Main Outcomes and MeasuresThree outcomes were hospital length of stay, probability of discharge disposition to a facility, and probability of inpatient mortality. The prediction performance metric for length of stay was mean absolute error. Prediction performance metrics for discharge disposition and inpatient mortality were average precision, precision, recall, specificity, F1 score, and area under the receiver operating characteristic curve (AUROC). Calibration was evaluated using calibration plots. Shapley addictive explanations analysis and bee swarm plots facilitated model explainability analysis.ResultsThe Length of Stay, Disposition, Mortality (LDM) Injury Index (the model) comprised a multitask deep learning model trained, developed, and internally validated on a data set of 372 573 traumatic injury encounters (mean [SD] age = 68.7 [19.3] years, 56.6% female). The model used 176 potential injuries to output 3 interpretable outcomes: the predicted hospital length of stay, probability of discharge to a facility, and probability of inpatient mortality. For the external validation set, the ISS predicted length of stay with mean absolute error was 4.16 (95% CI, 4.13-4.20) days. Compared with the ISS, the model had comparable external validation set discrimination performance (facility discharge AUROC: 0.67 [95% CI, 0.67-0.68] vs 0.65 [95% CI, 0.65-0.66]; recall: 0.59 [95% CI, 0.58-0.61] vs 0.59 [95% CI, 0.58-0.60]; specificity: 0.66 [95% CI, 0.66-0.66] vs 0.62 [95%CI, 0.60-0.63]; mortality AUROC: 0.83 [95% CI, 0.81-0.84] vs 0.82 [95% CI, 0.82-0.82]; recall: 0.74 [95% CI, 0.72-0.77] vs 0.75 [95% CI, 0.75-0.76]; specificity: 0.81 [95% CI, 0.81-0.81] vs 0.76 [95% CI, 0.75-0.77]). The model had excellent calibration for predicting facility discharge disposition, but overestimated inpatient mortality. Explainability analysis found the inputs influencing model predictions matched intuition.Conclusions and RelevanceIn this cohort study using a limited number of injury patterns, the model quantified injury severity using 3 intuitive outcomes. Further study is required to evaluate the model at scale.

show abstract

The derivation of an International Classification of Diseases, Tenth Revision–based trauma-related mortality model using machine learning

Cited by 10 publications

References 27 publications

ICD-10 based machine learning models outperform the Trauma and Injury Severity Score (TRISS) in survival prediction

ICD-10 based machine learning models outperform the Trauma and Injury Severity Score (TRISS) in survival prediction

Machine learning-based modeling of acute respiratory failure following emergency general surgery operations

Development and Validation of a Model to Quantify Injury Severity in Real Time

Contact Info

Product

Resources

About