Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models

Ramspek, Chava; Teece, Lucy; Snell, Kym I E; Evans, Marie; Riley, Richard D; Smeden, Maarten van; Geloven, Nan van; Diepen, Merel van

doi:10.1093/ije/dyab256

Cited by 23 publications

(57 citation statements)

References 50 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 18 Not accounting for competing risks can lead to an overestimation of the bleeding risk, certainly in a population of patients with cancer. 23 A limitation of this study is that both misclassification of the outcome and of predictors could have occurred. Regarding the predictors, most predictor definitions used in our validation closely resembled the definitions used in the development studies, however, some minor differences could not be avoided.…”

Section: Discussionmentioning

confidence: 94%

External validation and updating of prediction models of bleeding risk in patients with cancer receiving anticoagulants

et al. 2023

View full text Add to dashboard Cite

ObjectivePatients with cancer are at increased bleeding risk, and anticoagulants increase this risk even more. Yet, validated bleeding risk models for prediction of bleeding risk in patients with cancer are lacking. The aim of this study is to predict bleeding risk in anticoagulated patients with cancer.MethodsWe performed a study using the routine healthcare database of the Julius General Practitioners’ Network. Five bleeding risk models were selected for external validation. Patients with a new cancer episode during anticoagulant treatment or those initiating anticoagulation during active cancer were included. The outcome was the composite of major bleeding and clinically relevant non-major (CRNM) bleeding. Next, we internally validated an updated bleeding risk model accounting for the competing risk of death.ResultsThe validation cohort consisted of 1304 patients with cancer, mean age 74.0±10.9 years, 52.2% males. In total 215 (16.5%) patients developed a first major or CRNM bleeding during a mean follow-up of 1.5 years (incidence rate; 11.0 per 100 person-years (95% CI 9.6 to 12.5)). The c-statistics of all selected bleeding risk models were low, around 0.56. Internal validation of an updated model accounting for death as competing risk showed a slightly improved c-statistic of 0.61 (95% CI 0.54 to 0.70). On updating, only age and a history of bleeding appeared to contribute to the prediction of bleeding risk.ConclusionsExisting bleeding risk models cannot accurately differentiate bleeding risk between patients. Future studies may use our updated model as a starting point for further development of bleeding risk models in patients with cancer.

show abstract

Section: Discussionmentioning

confidence: 94%

External validation and updating of prediction models of bleeding risk in patients with cancer receiving anticoagulants

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Harrell's C-index was calculated as a measure of discrimination, using a prediction horizon of 10 years. For the outcomes KFRT, cardiovascular death, and MACE, we accounted for the competing risk of (non-cardiovascular) death by setting the follow-up time to the administrative censoring time for patients who experienced the competing event [ 27 ]. In further analyses, we adjusted hazard ratios for age and sex, as well as comorbidities and medication use, to make results comparable with previous studies [ 26 , 28 ].…”

Section: Methodsmentioning

confidence: 99%

Removing race from the CKD-EPI equation and its impact on prognosis in a predominantly White European population

Coresh

Grams

et al. 2022

Nephrology Dialysis Transplantation

Self Cite

View full text Add to dashboard Cite

Background While American nephrology societies recommend using the 2021 CKD-EPI eGFR equation without a Black race coefficient, it is unknown how this would impact disease distribution, prognosis and kidney failure risk prediction in predominantly White non-US populations. Methods We studied 1.6 million Stockholm adults with serum/plasma creatinine measurements between 2007–2019. We calculated changes in eGFR and reclassification across KDIGO GFR categories when changing from the 2009 to 2021 CKD-EPI equation; estimated associations between eGFR and the clinical outcomes kidney failure with replacement therapy (KFRT), (cardiovascular) mortality and major adverse cardiovascular events using Cox regression; and investigated prognostic accuracy (discrimination and calibration) of both equations within the Kidney Failure Risk Equation. Results Compared with the 2009 equation, the 2021 equation yielded a higher eGFR by a median (IQR) of 3.9 (2.9–4.8) ml/min/1.73m2, which was larger at older age and for men. Consequently, 9.9% of the total population and 36.2% of the population with CKD G3a-G5 was reclassified to a higher eGFR category. Reclassified individuals exhibited a lower risk of KFRT, but higher risks of all-cause/cardiovascular death and major adverse cardiovascular events, compared with non-reclassified participants of similar eGFR. eGFR by both equations strongly predicted study outcomes, with equal discrimination and calibration for the Kidney Failure Risk Equation. Conclusions Implementing the 2021 CKD-EPI equation in predominantly White European populations would raise eGFR by a modest amount (larger at older age and men) and shift a major proportion of CKD patients to a higher eGFR category. eGFR by both equations strongly predicted outcomes.

show abstract

“…A more realistic goal is to assess if the curve is close to the diagonal indicating moderate calibration. We plotted calibration curves estimated by smoothed local linear regression (loess) based on pseudovalues obtained from cumulative incidence estimates that account for the competing risk of death [33,34,37].…”

Section: Predictive Performance Of Kfrementioning

confidence: 99%

“…Employing robust statistical methods and sound analytical techniques, the study appropriately assesses the performance of KFRE in predicting kidney failure while considering the competing risk of death without kidney failure. This approach helps avoid overestimation of the observed risk and reduces bias in performance assessment, as opposed to solely relying on Cox methods, which were originally considered in the study protocol [33,34,50,53].…”

Section: Strengths and Limitations Of This Studymentioning

confidence: 99%

Kidney Failure Prediction: Multicenter External Validation of KFRE Model in Patients with CKD Stages 3-4 in Peru

Bravo-Zúñiga

Chávez-Gómez

Soto-Becerra

2023

Preprint

View full text Add to dashboard Cite

Objectives: To externally validate the 4-variable Kidney Failure Risk Equation (KFRE) in the Peruvian population for predicting kidney failure at 2 and 5 years. Design: A retrospective cohort study. Setting: 17 primary care centers from the Health's Social Security of Peru. Participants: Patients older than 18 years, diagnosed with chronic kidney disease (CKD) stage 3a-3b-4 and 3b-4, between January 2013 and December 2017. Patients were followed until they developed kidney failure, died, were lost, or ended the study (December 31, 2019), whichever came first. Primary and Secondary Outcome Measures: Performance of the KFRE model was assessed based on discrimination and calibration measures considering the competing risk of death. Results: We included 7519 patients in stages 3a-4 and 2,798 patients in stages 3b-4. The estimated cumulative incidence of kidney failure, accounting for competing event of death, at two years and five years was 1.52% and 3.37% in stages 3a-4 and 3.15% and 6.86% in stages 3b-4. KFRE discrimination at 2 and 5 years was high, with Time-Dependent Area Under the Curve (AUC-td) and C-index > 0.8 for all populations. Regarding calibration in-the-large, the Observed-to-Expected (O/E) ratio and the calibration intercept indicated that KFRE underestimates the overall risk at two years and overestimates it at 5-years in all populations. Conclusions: The 4-variable KFRE models have good discrimination but poor calibration in the Peruvian population. The model underestimates the risk of kidney failure in the short term and overestimates it in the long term. Further research should focus on updating or recalibrating the KFRE model to better predict kidney failure in the Peruvian context before recommending its use in clinical practice.

show abstract

Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models

Cited by 23 publications

References 50 publications

External validation and updating of prediction models of bleeding risk in patients with cancer receiving anticoagulants

External validation and updating of prediction models of bleeding risk in patients with cancer receiving anticoagulants

Removing race from the CKD-EPI equation and its impact on prognosis in a predominantly White European population

Kidney Failure Prediction: Multicenter External Validation of KFRE Model in Patients with CKD Stages 3-4 in Peru

Contact Info

Product

Resources

About