Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity

Abstract

ObjectiveTo explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.Study design and settingUsing individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV).ResultsPrior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions.ConclusionsPredictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully.Trial registration numberPROSPERO id: CRD42018088129.

Anticholinergic burden measures, symptoms, and fall-associated risk in older adults with polypharmacy: Development and validation of a prognostic model

“…Sensitivity analysis indicated that intervention status had an impact on the model’s outcome. A general challenge to external model validation in different study cohorts has already been identified in the prediction of hospital admissions in PROPERmed [ 32 ]. In our case, this may be partly explained by differences in the inclusion criteria and in the baseline risk of patients from the development and validation datasets.…”

“…Data from the binational PROPERmed database were used in model development and validation. In PROPERmed, individual participant data from five German and Dutch cluster-randomized controlled trials (cRCTs) were combined for pooled modelling purposes [30][31][32]. For this study, variables on ACh symptoms, a history of falls, and cognitive function were additionally taken from the studies.…”