Evaluating survival model performance: a graphical approach

Mandel, Micha; Galai, Noya; Simchen, Elisheva

doi:10.1002/sim.2089

Cited by 7 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A slope of 1.0 indicates good model calibration and a p-value >0.05 suggests that the slope is not statistically different from 1.0 (14). Model discrimination, or the ability to separate patients with higher and lower risk of CHD events, was assessed using a time-dependent c-statistic obtained from applying the model coefficients to the validation data set (15).…”

Section: Discussionmentioning

confidence: 99%

“…Israni et al Continued. Discrimination is the ability of the model to separate patients with a higher risk of coronary heart disease events from those with a lower risk; this was assessed using a time-dependent c-statistic obtained from applying the model coefficients to the validation data set (15). We used the shrinkage factor to modify the parameter estimates prior to calculating the time-dependent c-statistic in the validation data set.…”

Section: Predicting Posttransplant Chdmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Coronary Heart Disease after Kidney Transplantation: Patient Outcomes in Renal Transplantation (PORT) Study

Israni

Snyder

Skeans

et al. 2010

American Journal of Transplantation

166

127

View full text Add to dashboard Cite

Traditional risk factors do not adequately explain coronary heart disease (CHD) risk after kidney transplantation. We used a large, multicenter database to compare traditional and nontraditional CHD risk factors, and to develop risk-prediction equations for kidney transplant patients in standard clinical practice. We retrospectively assessed risk factors for CHD (acute myocardial infarction, coronary artery revascularization or sudden death) in 23 575 adult kidney transplant patients from 14 transplant centers worldwide. The CHD cumulative incidence was 3.1%, 5.2% and 7.6%, at 1, 3 and 5 years posttransplant, respectively. In separate Cox proportional hazards analyses of CHD in the first posttransplant year (predicted at time of transplant), and predicted within 3 years after a clinic visit occurring in posttransplant years 1-5, important risk factors included pretransplant diabetes, new onset posttransplant diabetes, prior pre-and posttransplant cardiovascular disease events, estimated glomerular filtration rate, delayed graft function, acute rejection, age, sex, race and duration of pretransplant end-stage kidney disease. The risk-prediction equations performed well, with the time-dependent c-statistic greater than 0.75. Traditional risk factors (e.g. hypertension, dyslipidemia and cigarette smoking) added little additional predictive value. Thus, transplant-related risk factors, particularly those linked to graft function, explain much of the variation in CHD after kidney transplantation. (4,5), and several nontraditional risk factors are reported to be associated with CHD after kidney transplantation (2,4,(6)(7)(8)(9)(10)(11)(12) Materials and Methods the discriminatory ability of the prediction model was not significantly improved by adding these Framingham risk factors and the prediction model performed much better than the Framingham Heart Study equation (Figure 2). We also examined CHD risk in the subset of patients from the 7 (of 14) 340 American Journal of Transplantation 2010; 10: 338-353Predicting Posttransplant CHD Figure 1: Predicted probability of coronary heart disease (dark line) with 95% confidence interval (lighter lines), and distribution of risk scores in the Patient Outcomes after Renal Transplant population (bars). (Panel A) Prediction of coronary heart disease in the first year posttransplant from variables available at the time of transplant (Table 1). (Panel B) Prediction of coronary heart disease in the first year posttransplant from variables available at the time of transplant and during the first week after transplant (Table 2). (Panel C) Prediction of coronary heart disease within 3 years after a visit occurring 1-5 years posttransplant (Table 3). study centers that routinely collected information on smoking status at the time of transplant (n = 9785 with 361 CHD events; Table A.4). Smoking status did not independently predict CHD, significantly affect which variables predicted CHD events, or affect the discriminatory ability of the prediction model (data not shown). Smokers were...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Predicting Posttransplant Chdmentioning

confidence: 99%

Predicting Coronary Heart Disease after Kidney Transplantation: Patient Outcomes in Renal Transplantation (PORT) Study

Israni

Snyder

Skeans

et al. 2010

American Journal of Transplantation

166

127

View full text Add to dashboard Cite

show abstract

“…Outliers and influence points were identified by checking deviance residuals and difference in betas (dfbetas). The performance of the models was assessed with the time-dependent C-statistic 23. Similar to the area under the receiver operator curve, it describes the probability that the model will assign the higher mortality risk to the patient who actually died as compared to the patient who remained alive or was censored.…”

Section: Methodsmentioning

confidence: 99%

Predicting Mortality in Incident Dialysis Patients: An Analysis of the United Kingdom Renal Registry

Wagner

Ansell

Kent

et al. 2011

American Journal of Kidney Diseases

111

View full text Add to dashboard Cite

Background The risk of death in dialysis patients remains high, but varies significantly among patients. No prediction tool is widely used in current clinical practice. We aimed to predict long-term mortality in incident dialysis patients with easily obtainable variables. Study Design Prospective nationwide multicenter cohort study in the United Kingdom (UK Renal Registry); Models were developed using Cox proportional hazards. Setting and Participants Patients initiating hemodialysis or peritoneal dialysis between 2002 and 2004, who survived at least three months on dialysis treatment, were followed for three years. Analyses were restricted to subjects in whom information on comorbid conditions and laboratory measurements were available (n=5447). The dataset was divided into datasets for model development (n=3631, training) and validation (n=1816) by random selection. Predictors Basic patient characteristics, comorbidity and laboratory variables. Outcomes All cause mortality censored for kidney transplant, recovery of kidney function, and loss to follow-up. Results In the training dataset, 1078 patients (29.7%) died within the observation period. The final model of the training dataset included patient characteristics (age, race, primary kidney disease, treatment modality), comorbidities (diabetes, history of cardiovascular disease, smoking) and laboratory variables (hemoglobin, serum albumin, creatinine, calcium) and reached a C-statistic of 0.75 (95% CI, 0.73–0.77) and could accurately discriminate between patients with low (6%), intermediate (19%), high (33%) and very high (59%) mortality risk. The model was further applied to the validation dataset and achieved a C-statistic of 0.73 (95% CI, 0.71–0.76). Limitations Number of missing comorbidity data and lack of an external validation dataset. Conclusions Basic patient characteristics, comorbidity and laboratory variables can predict three-year mortality in incident dialysis patients with sufficient accuracy. Identification of subgroups of patients according to mortality risk can guide future research and subsequently target treatment decisions in individual patients.

show abstract

“…Because model development is not the focus of this paper, we provide here only a limited overview of our approach. For a more complete discussion of model development and evaluation methodology see Klein and Moeschberger [11], Harrell [24], and Mandel [29].…”

Section: Model Developmentmentioning

confidence: 99%

Empirical estimation of life expectancy from large clinical trials: Use of left‐truncated, right‐censored survival analysis methodology

Nelson¹,

Sun²,

Tsiatis³

et al. 2008

Statistics in Medicine

View full text Add to dashboard Cite

In the current era of ever-increasing health care costs, economic analyses are an essential component in the comprehensive evaluation of new medical interventions. Cost-effectiveness analysis (CEA)--the most common form of economic analysis used in medicine--aids policy-makers in determining how to allocate finite health care dollars among possible alternative therapies. CEA relates the incremental benefits of a new technology to its incremental costs in a cost-effectiveness (CE) ratio. Although the generally agreed-upon standard of presentation for the CE ratio is the lifetime perspective (incremental lifetime cost to add one life year), this perspective presents an obvious challenge to the statistical analyst. Most large clinical trials collect limited follow-up data, and yet their findings form the basis of therapeutic recommendations that often extend far beyond the limits of the empirical data. Although clinical practice guidelines do not yet require explicit modeling to examine the long-term implications of their recommendations, health policy analyses routinely rely upon such extrapolations. This paper describes methods for using empirical patient-level data to extrapolate survival in large clinical trials and cohorts beyond a limited follow-up period in which most patients remain alive in order to estimate the entire survival distribution for a cohort of patients. We accomplish this task through a novel combination of models that estimate the hazard rate not only as a function of time but also as a function of patient age. Extrapolation of survival beyond a limited time frame is made possible by capitalizing on the extensive latitude of survival information available across the range of ages represented in the data. Variations in approach are presented, and issues arising in these analyses are discussed. The proposed methodology is developed, applied, and evaluated in both a large clinical trial cohort with 5-year follow-up on over 23,000 patients and a large observational database with long-term follow-up on over 4000 patients.

show abstract

Evaluating survival model performance: a graphical approach

Cited by 7 publications

References 24 publications

Predicting Coronary Heart Disease after Kidney Transplantation: Patient Outcomes in Renal Transplantation (PORT) Study

Predicting Coronary Heart Disease after Kidney Transplantation: Patient Outcomes in Renal Transplantation (PORT) Study

Predicting Mortality in Incident Dialysis Patients: An Analysis of the United Kingdom Renal Registry

Empirical estimation of life expectancy from large clinical trials: Use of left‐truncated, right‐censored survival analysis methodology

Contact Info

Product

Resources

About