External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis

Abstract

Background Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting. Methods IPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis. Results Of 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model’s calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%. Conclusions The evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice. Trial registration PROSPERO ID: CRD42015029349.

Prediction and prevention of preeclampsia by physicians in Brazil: An original study

“…It is important to highlight that, although it was not the aim of this study, combined prediction models have shown moderate predictive performance, with important limitations such as heterogeneity of the populations studied, low reproducibility of the methods used (mainly uterine artery Doppler), and a substantial lack of external validation. Evidence to support the use of these PE prediction models in clinical decision-making is limited and their predictive performance must be examined and validated locally before they can be considered for use in clinical practice ( 28 ).…”

Detecting Preeclampsia Severity Using Maternal-Obstetrical Characteristics and Complete Blood Cell Counts

“…10 In this regard, alternative predictive tools have been designed to improve the accuracy of diagnosis of the severity of preeclampsia. 11 , 12 …”

“…10 In this regard, alternative predictive tools have been designed to improve the accuracy of diagnosis of the severity of preeclampsia. 11,12 Our study was focused on hypertensive pregnancies immediately before delivery that corresponds well with those without routine prenatal visits during the pandemic. In order to adapt to the recent change in ACOG treatment guidelines and the wavering blood pressure or proteinuria accuracy, we aimed to use an efficient alternative method of combining maternal-obstetrical characteristics (MOCs) and complete blood cell counts (CBCs) with different red blood cell (RBC) indices 13 to evaluate and further confirm PE severity immediately.…”

Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis

“…FIGURE 7 Calibration plots for models predicting any-onset pre-eclampsia using first-trimester clinical characteristics and biochemical markers in data sets with > 100 outcome events. (a) Model 4115 in POP; 161 (b) model 5 128 in POP; 161 (c) model 6147 in St George's; 163 and (d) model 6147 in POP.161 Reproduced from Snell et al52 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material.…”

“…This figure contains information from several sources 115,147,[229][230][231][232][233][234][235][236]238 Reproduced from Snell et al52 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material.…”