Reporting and risk of bias of prediction models based on machine learning methods in preterm birth: A systematic review

Abstract

Introduction There was limited evidence on the quality of reporting and methodological quality of prediction models using machine learning methods in preterm birth. This systematic review aimed to assess the reporting quality and risk of bias of a machine learning‐based prediction model in preterm birth. Material and methods We conducted a systematic review, searching the PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure, China Biology Medicine disk, VIP Database, and WanFang Data from inception to September 27, 2021. Studies that developed (validated) a prediction model using machine learning methods in preterm birth were included. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement and Prediction model Risk of Bias Assessment Tool (PROBAST) to evaluate the reporting quality and the risk of bias of included studies, respectively. Findings were summarized using descriptive statistics and visual plots. The protocol was registered in PROSPERO (no. CRD 42022301623). Results Twenty‐nine studies met the inclusion criteria, with 24 development‐only studies and 5 development‐with‐validation studies. Overall, TRIPOD adherence per study ranged from 17% to 79%, with a median adherence of 49%. The reporting of title, abstract, blinding of predictors, sample size justification, explanation of model, and model performance were mostly poor, with TRIPOD adherence ranging from 4% to 17%. For all included studies, 79% had a high overall risk of bias, and 21% had an unclear overall risk of bias. The analysis domain was most commonly rated as high risk of bias in included studies, mainly as a result of small effective sample size, selection of predictors based on univariable analysis, and lack of calibration evaluation. Conclusions Reporting and methodological quality of machine learning‐based prediction models in preterm birth were poor. It is urgent to improve the design, conduct, and reporting of such studies to boost the application of machine learning‐based prediction models in preterm birth in clinical practice.

TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

“…However, systematic reviews evaluating studies of prediction models have shown that they are often poorly conducted (including deficiencies in study design or data collection 37 38 ); use poor methodology 37 38 ; are incompletely reported with key details missing 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 ; are consequently at high risk of bias 41 49 55 56 57 ; rarely adhere to open science practices, 58 and are susceptible to overinterpretation or so-called spin. 59 60 These deficiencies cast considerable doubt on models’ usefulness and safety, and raises concerns about their potential to create or widen healthcare disparities.…”

Prediction of spontaneous preterm birth using supervised machine learning on metabolomic data: A case–cohort study

“…The comparison of the present study with previous studies is difficult because of variation in the design and conduct of the published studies, the sample types, gestational ages when samples were drawn, metabolomics platforms used for analysis, and the statistical or machine learning approaches used. Some published studies have combined sPTB and iPTB as a single outcome, and the reporting of these studies has typically been inadequate, 29 making quantitative comparison difficult. Previous studies have treated sPTB as a binary outcome rather than using a time‐to‐delivery approach, mainly due to restrictions in study design.…”

“…small sample size and selection of predictors based on univariable analysis) and reporting of machine learning-based prediction models for PTB have generally been poor. 29 The aim of the present study was to identify and validate metabolites predictive of sPTB using multiple machine learning methods and sequential maternal serum samples from a large, prospective pregnancy cohort. This study addressed the limitations of previously published PTB models by using rigorous methodology (feature selection based on multiple models, use of multiple temporally separated sampling points, internal validation).…”

“…However, according to a recent systematic review, the methodological quality (e.g. small sample size and selection of predictors based on univariable analysis) and reporting of machine learning‐based prediction models for PTB have generally been poor 29 . The aim of the present study was to identify and validate metabolites predictive of sPTB using multiple machine learning methods and sequential maternal serum samples from a large, prospective pregnancy cohort.…”

Evaluating the Efficacy of Cervical Tactile Ultrasound Technique as a Predictive Tool for Spontaneous Preterm Birth

“…Despite the substantial progress in understanding the pathophysiology of preterm birth, predicting and preventing it remains a significant challenge [ 6 ] [ 7 ] [ 8 ]. Several conventional methods have been employed to identify women at risk of sPTB, including maternal history, clinical examination, biochemical markers, and ultrasound-based cervical length measurement [ 9 ] [ 10 ].…”