Protocol for the development and validation of a risk prediction model for stillbirths from 35 weeks gestation in Australia

Sexton, Jessica; Coory, Michael; Kumar, Sailesh; Smith, Gordon Cs; Gordon, Adrienne; Chambers, Georgina M.; Pereira, Gavin; Raynes‐Greenow, Camille; Hilder, Lisa; Middleton, Philippa; Bowman, Anneka; Lieske, Scott N.; Warrilow, Kara; Morris, Jonathan M.; Ellwood, David; Flenady, Vicki

doi:10.1186/s41512-020-00089-w

Cited by 3 publications

(3 citation statements)

References 48 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the UK, the stillbirth rate was largely unchanged from 2000 to 2015, and, in 2017, the rate was one of the highest in Europe, at 4.2 stillbirths/1000 births [3][4][5] . Prediction and individualization of risk remain key priorities for stillbirth research 6,7 , because accurate identification of women at high risk of stillbirth can guide decisions on the need for closer surveillance and timing of delivery in order to prevent fetal death. A recent review that identified existing prediction models for stillbirth reported that none had been validated externally 8 .…”

Section: Introductionmentioning

confidence: 99%

External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta‐analysis

Allotey

Whittle

Snell

et al. 2022

Ultrasound in Obstet & Gyne

View full text Add to dashboard Cite

Objective Stillbirth is a potentially preventable complication of pregnancy. Identifying women at high risk of stillbirth can guide decisions on the need for closer surveillance and timing of delivery in order to prevent fetal death. Prognostic models have been developed to predict the risk of stillbirth, but none has yet been validated externally. In this study, we externally validated published prediction models for stillbirth using individual participant data (IPD) meta‐analysis to assess their predictive performance. Methods MEDLINE, EMBASE, DH‐DATA and AMED databases were searched from inception to December 2020 to identify studies reporting stillbirth prediction models. Studies that developed or updated prediction models for stillbirth for use at any time during pregnancy were included. IPD from cohorts within the International Prediction of Pregnancy Complications (IPPIC) Network were used to validate externally the identified prediction models whose individual variables were available in the IPD. The risk of bias of the models and cohorts was assessed using the Prediction study Risk Of Bias ASsessment Tool (PROBAST). The discriminative performance of the models was evaluated using the C‐statistic, and calibration was assessed using calibration plots, calibration slope and calibration‐in‐the‐large. Performance measures were estimated separately in each cohort, as well as summarized across cohorts using random‐effects meta‐analysis. Clinical utility was assessed using net benefit. Results Seventeen studies reporting the development of 40 prognostic models for stillbirth were identified. None of the models had been previously validated externally, and the full model equation was reported for only one‐fifth (20%, 8/40) of the models. External validation was possible for three of these models, using IPD from 19 cohorts (491 201 pregnant women) within the IPPIC Network database. Based on evaluation of the model development studies, all three models had an overall high risk of bias, according to PROBAST. In the IPD meta‐analysis, the models had summary C‐statistics ranging from 0.53 to 0.65 and summary calibration slopes ranging from 0.40 to 0.88, with risk predictions that were generally too extreme compared with the observed risks. The models had little to no clinical utility, as assessed by net benefit. However, there remained uncertainty in the performance of some models due to small available sample sizes. Conclusions The three validated stillbirth prediction models showed generally poor and uncertain predictive performance in new data, with limited evidence to support their clinical application. The findings suggest methodological shortcomings in their development, including overfitting. Further research is needed to further validate these and other models, identify stronger prognostic factors and develop more robust prediction models. © 2021 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gyne...

show abstract

Section: Introductionmentioning

confidence: 99%

External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta‐analysis

Allotey

Whittle

Snell

et al. 2022

Ultrasound in Obstet & Gyne

View full text Add to dashboard Cite

show abstract

“…ROC analysis assesses the model's ability to classify observations by plotting sensitivity versus 1-specificity (25). The area under the curve (AUC) values were categorized as follows: high accuracy (0.9 < AUC-ROC ≤1), moderate accuracy (0.7 < AUC-ROC ≤0.9), and low accuracy (0.5 < AUC-ROC ≤0.7) (26). Calibration plots were used to assess the relationship between predicted survival probability and observed survival (27).…”

Section: Nomogram Construction and Validationmentioning

confidence: 99%

An online survival predictor in glioma patients using machine learning based on WHO CNS5 data

Zheng

et al. 2023

Front. Neurol.

View full text Add to dashboard Cite

BackgroundThe World Health Organization (WHO) CNS5 classification system highlights the significance of molecular biomarkers in providing meaningful prognostic and therapeutic information for gliomas. However, predicting individual patient survival remains challenging due to the lack of integrated quantitative assessment tools. In this study, we aimed to design a WHO CNS5-related risk signature to predict the overall survival (OS) rate of glioma patients using machine learning algorithms.MethodsWe extracted data from patients who underwent an operation for histopathologically confirmed glioma from our hospital database (2011–2022) and split them into a training and hold-out test set in a 7/3 ratio. We used biological markers related to WHO CNS5, clinical data (age, sex, and WHO grade), and prognosis follow-up information to identify prognostic factors and construct a predictive dynamic nomograph to predict the survival rate of glioma patients using 4 kinds machine learning algorithms (RF, SVM, XGB, and GLM).ResultsA total of 198 patients with complete WHO5 molecular data and follow-up information were included in the study. The median OS time of all patients was 29.77 [95% confidence interval (CI): 21.19–38.34] months. Age, FGFR2, IDH1, CDK4, CDK6, KIT, and CDKN2A were considered vital indicators related to the prognosis and OS time of glioma. To better predict the prognosis of glioma patients, we constructed a WHO5-related risk signature and nomogram. The AUC values of the ROC curves of the nomogram for predicting the 1, 3, and 5-year OS were 0.849, 0.835, and 0.821 in training set, and, 0.844, 0.943, and 0.959 in validation set. The calibration plot confirmed the reliability of the nomogram, and the c-index was 0.742 in training set and 0.775 in validation set. Additionally, our nomogram showed a superior net benefit across a broader scale of threshold probabilities in decision curve analysis. Therefore, we selected it as the backend for the online survival prediction tool (Glioma Survival Calculator, https://who5pumch.shinyapps.io/DynNomapp/), which can calculate the survival probability for a specific time of the patients.ConclusionAn online prognosis predictor based on WHO5-related biomarkers was constructed. This therapeutically promising tool may increase the precision of forecast therapy outcomes and assess prognosis.

show abstract

“…Although the study successfully motivated further endeavour for the development of such models in India, several design and reporting limitations hamper both inference and application. I propose the following methodological considerations for future studies on the topic based on our team’s experience with one of the largest studies ( n > 5,500 stillbirths) on stillbirth prediction to date, published in Nature Scientific Reports [ 3 ]; externally validated studies on preterm birth [ 4 , 5 ]; and a recent registered protocol [ 6 ].…”

Section: Main Textmentioning

confidence: 99%

Prediction Models for Adverse Pregnancy Outcomes in India: Methodological Considerations for an Emerging Topic

Pereira

2022

J Obstet Gynecol India

Self Cite

View full text Add to dashboard Cite

Stillbirth is over-represented in lower and lower-middle-income countries and understandably this has motivated greater research investment in the development of prediction models. Prediction is particularly challenging for pregnancy outcomes because only part of the population is represented in observational research. Notably, unrecognised pregnancies and miscarriages are typically excluded from the development of prediction models and the consequences of such selection are not well understood. Other methodological challenges in developing stillbirth prediction models are within the control of the researcher. Identifying whether the intended model is for aetiological explanation versus prediction, attainment of a sufficiently large representative sample, and internal and external validation are among such methodological considerations. These considerations are discussed in relation to a recently published study on prediction of stillbirth after 28 weeks of pregnancy for women with hypertensive disorders of pregnancy in India. The predictive ability of this model amounts to the flip of a coin. Future screening based on such a model may be expensive, increase psychological distress among patients and introduce additional iatrogenic perinatal morbidities from over-treatment. Future research should address the methodological considerations described in this article.

show abstract

Protocol for the development and validation of a risk prediction model for stillbirths from 35 weeks gestation in Australia

Cited by 3 publications

References 48 publications

External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta‐analysis

External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta‐analysis

An online survival predictor in glioma patients using machine learning based on WHO CNS5 data

Prediction Models for Adverse Pregnancy Outcomes in India: Methodological Considerations for an Emerging Topic

Contact Info

Product

Resources

About