Kidney Failure Prediction Models: A Comprehensive External Validation Study in Patients with Advanced CKD

Abstract

BackgroundVarious prediction models have been developed to predict the risk of kidney failure in patients with CKD. However, guideline-recommended models have yet to be compared head to head, their validation in patients with advanced CKD is lacking, and most do not account for competing risks.MethodsTo externally validate 11 existing models of kidney failure, taking the competing risk of death into account, we included patients with advanced CKD from two large cohorts: the European Quality Study (EQUAL), an ongoing European prospective, multicenter cohort study of older patients with advanced CKD, and the Swedish Renal Registry (SRR), an ongoing registry of nephrology-referred patients with CKD in Sweden. The outcome of the models was kidney failure (defined as RRT-treated ESKD). We assessed model performance with discrimination and calibration.ResultsThe study included 1580 patients from EQUAL and 13,489 patients from SRR. The average c statistic over the 11 validated models was 0.74 in EQUAL and 0.80 in SRR, compared with 0.89 in previous validations. Most models with longer prediction horizons overestimated the risk of kidney failure considerably. The 5-year Kidney Failure Risk Equation (KFRE) overpredicted risk by 10%–18%. The four- and eight-variable 2-year KFRE and the 4-year Grams model showed excellent calibration and good discrimination in both cohorts.ConclusionsSome existing models can accurately predict kidney failure in patients with advanced CKD. KFRE performed well for a shorter time frame (2 years), despite not accounting for competing events. Models predicting over a longer time frame (5 years) overestimated risk because of the competing risk of death. The Grams model, which accounts for the latter, is suitable for longer-term predictions (4 years).

The Effect of Age on Performance of the Kidney Failure Risk Equation in Advanced CKD

“…17 In an extension of these findings, a recent study by Ramspek et al directly compared the predictive performance of 11 kidney failure risk prediction models. 5 While the KFRE performed well within two years, it overestimated the risk of kidney failure over five years due to the competing risk of death. 5 Alternative risk prediction models that accounted for the competing risk of death, such as the Grams model, 22 were better suited to estimate kidney failure over longer time frames.…”

“…Prediction models with death treated as a censoring event provided kidney failure risk estimates that were 7% higher at 5 years and 19% higher at 10 years compared with models with death treated as a competing event. 17 In an extension of these findings, a recent study by Ramspek et al 5 directly compared the predictive performance of 11 kidney failure risk prediction models. Although the KFRE performed well within 2 years, it overestimated the risk of kidney failure over 5 years owing to the competing risk of death.…”

“…In order to provide individualized risk estimates for patients with CKD, a number of risk prediction models have been developed and integrated into clinical practice. 4,5 The most widely used prediction model, in part related to its simplicity of use, is the Kidney Failure Risk Equation (KFRE). 6 The KFRE has been extensively validated in a number of different patient populations and geographical settings.…”

“…Not accounting for the competing risk of death has been found to result in an overestimation of the risk of kidney failure particularly over longer periods of follow-up. 5 , 16 , 17 , 18 Given the direct relationship between age and mortality, this overestimation of kidney failure risk may be more relevant to elderly populations and less relevant to younger populations—a topic which, to our knowledge, has not been previously studied. Here, we evaluated the impact of age on the 4-variable KFRE performance in a large population of patients with advanced CKD.…”

Appraising prediction research: a guide and meta‐review on bias and applicability assessment using the Prediction model Risk Of Bias ASsessment Tool (PROBAST)

“…For example, models developed for late-stage chronic kidney disease (CKD) should be used with caution in early CKD, and vice-versa. 9 The second ROB subdomain, the predictors selection subdomain, consists of three signalling questions and covers the sources of bias that may arise due to the definition and measurement of the predictors. First, the predictors should be defined and assessed in the same way for all participants (e.g., an issue if the predictor 'body weight' was self-reported for some participants and measured for others).…”