SAPS 3 admission score: an external validation in a general intensive care population

Abstract

Objectives: To validate the SAPS 3 admission score in an independent general intensive care case mix and to compare its performances with the APACHE II and the SAPS II scores. Design: Cohort observational study. Setting: A 26-bed general ICU from a Tertiary University Hospital. Patients and participants: Eight hundred and fifty-one consecutive patients admitted to the ICU over an 8-month period. Of these patients, 49 were readmissions, leaving 802 patients for further analysis. Intervention: None. Measurements and results: APACHE II, SAPS II and SAPS 3 variables were prospectively collected; scores and their derived probability of death were calculated according to their original manuscript description. The discriminative power was assessed using the area under the ROC curve (AUROC) and calibration was verified with the Hosmer-Lemeshow goodness-of-fit test. The AUROC of the APACHE II model (AUROC = 0.823) was significantly lower than those of the SAPS II (AUROC = 0.850) and SAPS 3 models (AUROC = 0.854) (P = 0.038). The calibration of the APACHE II model (P = 0.037) and of the SAPS 3 global model (P = 0.035) appeared unsatisfactory. On the contrary, both SAPS II model and SAPS 3 model customised for Central and Western Europe had a good calibration. However, in our study case mix, SAPS II model tended to overestimate the probability of death. Conclusion: In this study, the SAPS 3 admission score and its prediction model customised for Central and Western Europe was more discriminative and better calibrated than APACHE II, but it was not significantly better than the SAPS II.

Calibration strategies to validate predictive models: is new always better?

“…Similarly, a common finding in most external validation studies is that models show poor calibration, while discrimination is usually good for all of them [11][12][13]. Miscalibration of SAPS 3 has been already reported in several previous studies [11][12][13][14][15][16][17]. Although good performance of SAPS 3 has been shown in some general settings [18,19], in most instances SAPS 3 presents better calibration in specific subsets of critically ill patients [16,18,[20][21][22][23][24][25][26].…”

Year in review in Intensive Care Medicine, 2008: III. Paediatrics, Ethics, outcome research and critical care organization, sedation, pharmacology and miscellanea

“…As for all severity of illness systems, external validation studies are needed to proof the prognostic performance in settings different than the one the score has been developed from. Ledoux et al [49] undertook a prospective study in their institution to evaluate the performance of SAPS 3 and to compare it to two other systems, namely APACHE II and SAPS II. They included 802 consecutively to the ICU admitted patients over an 8-month period.…”

Mortality Prediction in the ICU Based on MIMIC-II Results from the Super ICU Learner Algorithm (SICULA) Project

“…However, since first being published, they have been modified several times in order to improve their predictive performance [6][7][8][9][10][11]. Despite these extensions of SAPS, predicted hospital mortality remains generally overestimated [8,9,[12][13][14]. As an illustration, Poole et al [9] compared the SAPS II and the SAPS3 performance in a cohort of more than 28,000 admissions to 10 different Italian ICUs.…”