External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom

Shanahan, Thomas; Fuller, Gordon; Sheldon, Trevor; Turton, Emily; Quilty, Fionn Michael Appleton; Marincowitz, Carl

doi:10.1016/j.injury.2021.01.039

Cited by 7 publications

(5 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This means that the sensitivity is the probability that a patient with an ISS ≥ 16 is sent to an MTC and the specificity is the probability that a patient with an ISS < 16 is sent to a local hospital. A recent analysis of a Dutch triage tool in an English data set produced a received-operator curve that would result in similar triage tool sensitivity and specificity as observed by Newgard et al [11].…”

Section: Interventionssupporting

confidence: 58%

An economic evaluation of triage tools for patients with suspected severe injuries in England

et al. 2022

View full text Add to dashboard Cite

Background Many health care systems triage injured patients to major trauma centres (MTCs) or local hospitals by using triage tools and paramedic judgement. Triage tools are typically assessed by whether patients with an Injury Severity Score (ISS) ≥ 16 go to an MTC and whether patients with an ISS < 16 are sent to their local hospital. There is a trade-off between sensitivity and specificity of triage tools, with the optimal balance being unknown. We conducted an economic evaluation of major trauma triage tools to identify which tool would be considered cost-effective by UK decision makers. Methods A patient-level, probabilistic, mathematical model of a UK major trauma system was developed. Patients with an ISS ≥ 16 who were only treated at local hospitals had worse outcomes compared to being treated in an MTC. Nine empirically derived triage tools, from a previous study, were examined so we assessed triage tools with realistic trade-offs between triage tool sensitivity and specificity. Lifetime costs, lifetime quality adjusted life years (QALYs), and incremental cost-effectiveness ratios (ICERs) were calculated for each tool and compared to maximum acceptable ICERs (MAICERs) in England. Results Four tools had ICERs within the normal range of MAICERs used by English decision makers (£20,000 to £30,000 per QALY gained). A low sensitivity (28.4%) and high specificity (88.6%) would be cost-effective at the lower end of this range while higher sensitivity (87.5%) and lower specificity (62.8%) was cost-effective towards the upper end of this range. These results were sensitive to the cost of MTC admissions and whether MTCs had a benefit for patients with an ISS between 9 and 15. Conclusions The cost-effective triage tool depends on the English decision maker’s MAICER for this health problem. In the usual range of MAICERs, cost-effective prehospital trauma triage involves clinically suboptimal sensitivity, with a proportion of seriously injured patients (at least 10%) being initially transported to local hospitals. High sensitivity trauma triage requires development of more accurate decision rules; research to establish if patients with an ISS between 9 and 15 benefit from MTCs; or, inefficient use of health care resources to manage patients with less serious injuries at MTCs.

show abstract

Section: Interventionssupporting

confidence: 58%

An economic evaluation of triage tools for patients with suspected severe injuries in England

et al. 2022

View full text Add to dashboard Cite

show abstract

“…37 The model was also externally validated in a cohort of patients from the UK, and the AUROC was 0.75, while undertriage remained at 17% with an overtriage of 50%. The authors concluded that the model did not meet the ACSCOT recommendations, 38 and it is unclear how this model would perform in the USA. The original model was based on ISS ≥16, which is not an accurate predictor of severe injury.…”

Section: Discussionmentioning

confidence: 99%

Prospective validation of a hospital triage predictive model to decrease undertriage: an EAST multicenter study

Biesboer,

Pokrzywa,

Karam

et al. 2024

Trauma Surg Acute Care Open

View full text Add to dashboard Cite

BackgroundTiered trauma team activation (TTA) allows systems to optimally allocate resources to an injured patient. Target undertriage and overtriage rates of <5% and <35% are difficult for centers to achieve, and performance variability exists. The objective of this study was to optimize and externally validate a previously developed hospital trauma triage prediction model to predict the need for emergent intervention in 6 hours (NEI-6), an indicator of need for a full TTA.MethodsThe model was previously developed and internally validated using data from 31 US trauma centers. Data were collected prospectively at five sites using a mobile application which hosted the NEI-6 model. A weighted multiple logistic regression model was used to retrain and optimize the model using the original data set and a portion of data from one of the prospective sites. The remaining data from the five sites were designated for external validation. The area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) were used to assess the validation cohort. Subanalyses were performed for age, race, and mechanism of injury.Results14 421 patients were included in the training data set and 2476 patients in the external validation data set across five sites. On validation, the model had an overall undertriage rate of 9.1% and overtriage rate of 53.7%, with an AUROC of 0.80 and an AUPRC of 0.63. Blunt injury had an undertriage rate of 8.8%, whereas penetrating injury had 31.2%. For those aged ≥65, the undertriage rate was 8.4%, and for Black or African American patients the undertriage rate was 7.7%.ConclusionThe optimized and externally validated NEI-6 model approaches the recommended undertriage and overtriage rates while significantly reducing variability of TTA across centers for blunt trauma patients. The model performs well for populations that traditionally have high rates of undertriage.Level of evidence2.

show abstract

“…Values close to one indicate perfect discrimination ability, while values close to 0.5 indicate poor discrimination ability. Second, calibration was assessed by multiple methods [ 16 , 17 ]. The ratio of observed versus expected number of events (O/E-ratio) was pooled after logarithmic transformation.…”

Section: Methodsmentioning

confidence: 99%

“…This ratio should be close to one if the model calibrates well in the validation dataset. Then, model calibration was assessed visually with a calibration plot (a scatter plot of predicted versus observed outcome probabilities) for which well-calibrated predictions will lie along the 45° line [ 16 , 17 ]. Finally, calibration was assessed by fitting a logistic regression model with the observed outcome as the dependent variable, and the log-odds of transformed model predictions as the independent variable.…”

Section: Methodsmentioning

confidence: 99%

External Validation of the Colon Life Nomogram for Predicting 12-Week Mortality in Dutch Metastatic Colorectal Cancer Patients Treated with Trifluridine/Tipiracil in Daily Practice

Hamers

Wensink

Smeden

et al. 2022

Cancers

View full text Add to dashboard Cite

Background: Predicting prognosis in refractory metastatic colorectal cancer (mCRC) patients is needed to guide decision making. The Colon Life nomogram was developed to predict 12-week mortality in refractory mCRC patients. The aim of this study is to validate the Colon Life nomogram in last line/refractory patients receiving trifluridine/tipiracil (FTD/TPI) in daily practice. Methods: The validation cohort consists of 150 QUALITAS study patients, an observational substudy of the Prospective Dutch CRC cohort, who were treated with FTD/TPI between 2016 and 2019. Model performance was assessed on discrimination, calibration, and clinical usefulness. The additional prognostic value of baseline quality of life (QoL) and thymidine kinase (TK1) expression in tissue was explored. Results: Of the 150 patients, 25 (16.7%) died within 12 weeks of starting FTD/TPI treatment. The C-statistic was 0.63 (95% C.I. 0.56–0.70). The observed/expected ratio was 0.52 (0.37–0.73). The calibration intercept and slope were −1.06 (−1.53 to −0.58) and 0.41 (0.01–0.81), respectively, which indicated overestimation of 12-week mortality by the nomogram. Decision curve analysis showed the nomogram did not yield a positive net benefit at clinically meaningful thresholds for predicted 12-week mortality. Addition of QoL to the nomogram improved the C-statistic to 0.85 (0.81–0.89). TK1 expression was associated with progression-free survival but not with overall survival. Conclusion: We demonstrated evident miscalibration of the Colon Life nomogram upon external validation, which hampers its use in clinical practice. We recommend conducting a study with a sufficiently large sample size to update the Colon Life nomogram or to develop a new model including QoL.

show abstract

External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom

Cited by 7 publications

References 40 publications

An economic evaluation of triage tools for patients with suspected severe injuries in England

An economic evaluation of triage tools for patients with suspected severe injuries in England

Prospective validation of a hospital triage predictive model to decrease undertriage: an EAST multicenter study

External Validation of the Colon Life Nomogram for Predicting 12-Week Mortality in Dutch Metastatic Colorectal Cancer Patients Treated with Trifluridine/Tipiracil in Daily Practice

Contact Info

Product

Resources

About