Diagnostic and prognostic prediction models

Abstract

To cite this article: Hendriksen JMT, Geersing GJ, Moons KGM, de Groot JAH. Diagnostic and prognostic prediction models. J Thromb Haemost 2013; 11 (Suppl. 1): 129-41.Summary. Risk prediction models can be used to estimate the probability of either having (diagnostic model) or developing a particular disease or outcome (prognostic model). In clinical practice, these models are used to inform patients and guide therapeutic management. Examples from the field of venous thrombo-embolism (VTE) include the Wells rule for patients suspected of deep venous thrombosis and pulmonary embolism, and more recently prediction rules to estimate the risk of recurrence after a first episode of unprovoked VTE. In this paper, the three phases that are recommended before a prediction model may be used in daily practice are described: development, validation, and impact assessment. In the development phase, the focus is on model development commonly using a multivariable logistic (diagnostic) or survival (prognostic) regression analysis. The performance of the developed model is expressed by discrimination, calibration and (re-) classification. In the validation phase, the developed model is tested in a new set of patients using these same performance measures. This is important, as model performance is commonly poorer in a new set of patients, e.g. due to case-mix or domain differences. Finally, in the impact phase the ability of a prediction model to actually guide patient management is evaluated. Whereas in the development and validation phase single cohort designs are preferred, this last phase asks for comparative designs, ideally randomized designs; therapeutic management and outcomes after using the prediction model is compared to a control group not using the model (e.g. usual care).

Optimizing the G8 Screening Tool for Older Patients With Cancer: Diagnostic Performance and Validation of a Six-Item Version

“…This approach consisted of the initial selection of candidate items based on clinical reasoning, multivariate analyses to identify items conveying independent information, internal validation based on bootstrapping techniques to prevent overfitting [14], and reassessment of the model performance on a validation set population. Special attention was given to weights computation to obtain an easy-to-use tool while limiting the loss of information inevitably associated with rounding [15].…”

“…Special attention was given to weights computation to obtain an easy-to-use tool while limiting the loss of information inevitably associated with rounding [15]. Finally, we used multiple imputation at each step of model development to maintain an effective sample size and to control their potential influence on the final model [14].…”

“…We identified 22 candidate items for a modified G8, based on both the literature and clinical expertise, to avoid the overfitting to the training data set seen when item selection relies solely on statistical significance [14]. In addition to the G8 items (depression/dementia, body mass index [ …”

Evaluating the impact of prediction models: lessons learned, challenges, and recommendations

“…After repeated exposure to the predictions in a variety of index group patients, physicians may become better at estimating the probability in subsequent similar patients, even when these patients are part of the control group [8,9]. This likely dilutes the effectiveness and thus impact of the model use [47]. The effects of a learning curve may be minimized, though not completely prevented, by randomization at a cluster level, e.g.…”

How to Integrate Predictions in Outcomes in Planning Clinical Care

“…Prognostic tools are risk prediction models that estimate the probability of having a disease (diagnostic prediction model DPM), or for developing a certain outcome (prognosis prediction model PPM) based on various patient variables [17]. DPMs and PPMs can be tools to help guide SDM.…”