STUDY QUESTION What are the best-quality clinical prediction models in IVF (including ICSI) treatment to inform clinicians and their patients of their chance of success? SUMMARY ANSWER The review recommends the McLernon post-treatment model for predicting the cumulative chance of live birth over and up to six complete cycles of IVF. WHAT IS KNOWN ALREADY Prediction models in IVF have not found widespread use in routine clinical practice. This could be due to their limited predictive accuracy and clinical utility. A previous systematic review of IVF prediction models, published a decade ago and which has never been updated, did not assess the methodological quality of existing models nor provided recommendations for the best-quality models for use in clinical practice. STUDY DESIGN, SIZE, DURATION The electronic databases OVID MEDLINE, OVID EMBASE and Cochrane library were searched systematically for primary articles published from 1978 to January 2019 using search terms on the development and/or validation (internal and external) of models in predicting pregnancy or live birth. No language or any other restrictions were applied. PARTICIPANTS/MATERIALS, SETTING, METHODS The PRISMA flowchart was used for the inclusion of studies after screening. All studies reporting on the development and/or validation of IVF prediction models were included. Articles reporting on women who had any treatment elements involving donor eggs or sperm and surrogacy were excluded. The CHARMS checklist was used to extract and critically appraise the methodological quality of the included articles. We evaluated models’ performance by assessing their c-statistics and plots of calibration in studies and assessed correct reporting by calculating the percentage of the TRIPOD 22 checklist items met in each study. MAIN RESULTS AND THE ROLE OF CHANCE We identified 33 publications reporting on 35 prediction models. Seventeen articles had been published since the last systematic review. The quality of models has improved over time with regard to clinical relevance, methodological rigour and utility. The percentage of TRIPOD score for all included studies ranged from 29 to 95%, and the c-statistics of all externally validated studies ranged between 0.55 and 0.77. Most of the models predicted the chance of pregnancy/live birth for a single fresh cycle. Six models aimed to predict the chance of pregnancy/live birth per individual treatment cycle, and three predicted more clinically relevant outcomes such as cumulative pregnancy/live birth. The McLernon (pre- and post-treatment) models predict the cumulative chance of live birth over multiple complete cycles of IVF per woman where a complete cycle includes all fresh and frozen embryo transfers from the same episode of ovarian stimulation. McLernon models were developed using national UK data and had the highest TRIPOD score, and the post-treatment model performed best on external validation. LIMITATIONS, REASONS FOR CAUTION To assess the reporting quality of all included studies, we used the TRIPOD checklist, but many of the earlier IVF prediction models were developed and validated before the formal TRIPOD reporting was published in 2015. It should also be noted that two of the authors of this systematic review are authors of the McLernon model article. However, we feel we have conducted our review and made our recommendations using a fair and transparent systematic approach. WIDER IMPLICATIONS OF THE FINDINGS This study provides a comprehensive picture of the evolving quality of IVF prediction models. Clinicians should use the most appropriate model to suit their patients’ needs. We recommend the McLernon post-treatment model as a counselling tool to inform couples of their predicted chance of success over and up to six complete cycles. However, it requires further external validation to assess applicability in countries with different IVF practices and policies. STUDY FUNDING/COMPETING INTEREST(S) The study was funded by the Elphinstone Scholarship Scheme and the Assisted Reproduction Unit, University of Aberdeen. Both D.J.M. and S.B. are authors of the McLernon model article and S.B. is Editor in Chief of Human Reproduction Open. They have completed and submitted the ICMJE forms for Disclosure of potential Conflicts of Interest. The other co-authors have no conflicts of interest to declare. REGISTRATION NUMBER N/A
STUDY QUESTION Can we use prediction modelling to estimate the impact of coronavirus disease 2019 (COVID 19) related delay in starting IVF or ICSI in different groups of women? SUMMARY ANSWER Yes, using a combination of three different models we can predict the impact of delaying access to treatment by 6 and 12 months on the probability of conception leading to live birth in women of different age groups with different categories of infertility. WHAT IS KNOWN ALREADY Increased age and duration of infertility can prejudice the chances of success following IVF, but couples with unexplained infertility have a chance of conceiving naturally without treatment whilst waiting for IVF. The worldwide suspension of IVF could lead to worse outcomes in couples awaiting treatment, but it is unclear to what extent this could affect individual couples based on age and cause of infertility. STUDY DESIGN, SIZE, DURATION A population based cohort study based on national data from all licensed clinics in the UK obtained from the Human Fertilisation and Embryology Authority Register. Linked data from 9589 women who underwent their first IVF or ICSI treatment in 2017 and consented to the use of their data for research were used to predict livebirth numbers. PARTICIPANTS/MATERIALS, SETTING, METHODS Three prediction models were used to estimate the chances of livebirth associated with immediate treatment versus a delay of 6 and 12 months in couples about to embark on IVF or ICSI. MAIN RESULTS AND THE ROLE OF CHANCE We estimated that a 6-month delay would reduce livebirths by 0.4%, 2.4%, 5.7%, 9.5% and 11.8% in women aged <30, 30-35, 36-37, 38-39 and 40-42 years, respectively, while corresponding values associated with a delay of 12 months were 0.9%, 4.9%, 11.9%, 18.8% and 22.4%, respectively. In women with known causes of infertility, worst case (best case) predicted chances of livebirth after a delay of 6 months in women aged <30, 30-35, 36-37, 38-39 and 40-42 years varied between 31.6% (35.0%), 29.0% (31.6%), 23.1% (25.2%), 17.2% (19.4%) and 10.3% (12.3%) for tubal infertility and 34.3% (39.2%), 31.6% (35.3%) 25.2%(28.5%) 18.3% (21.3%), and 11.3% (14.1%) for male factor infertility. The corresponding values in those treated immediately were 31.7%, 29.8%, 24.5%, 19.0% and 11.7% for tubal factor and 34.4%, 32.4%, 26.7%, 20.2% and 12.8% in male factor infertility. In women with unexplained infertility the predicted chances of livebirth after a delay of 6 months followed by one complete IVF cycle were 41.0%, 36.6%, 29.4%, 22.4% and 15.1% in women aged <30, 30-35, 36-37, 38-39 and 40-42 years, respectively, compared to 34.9%, 32.5%, 26.9%, 20.7% and 13.2% in similar groups of women treated without any delay. The additional waiting period, which provided more time for spontaneous conception, was predicted to increase the relative number of babies born by 17.5%, 12.6%, 9.1%, 8.4% and 13.8%, in women aged <30, 30-35, 36-37, 38-39 and 40-42 years, respectively. A 12-month delay showed a similar pattern in all subgroups. LIMITATIONS, REASONS FOR CAUTION Major sources of uncertainty include the use of prediction models generated in different populations and the need for a number of assumptions. Although the models are validated and the bases for the assumptions are robust, it is impossible to eliminate the possibility of imprecision in our predictions. Therefore, our predicted live birth rates need to be validated in prospective studies to confirm their accuracy. WIDER IMPLICATIONS OF THE FINDINGS A delay in starting IVF reduces success rates in all couples. For the first time, we have shown that while this results in fewer babies in older women and those with a known cause of infertility, it has a less detrimental effect on couples with unexplained infertility, some of whom conceive naturally whilst waiting for treatment. Post COVID 19, clinics planning a phased return to normal clinical services should prioritise older women and those with a known cause of infertility. STUDY FUNDING/COMPETING INTEREST(S) No external funding was received for this study. B.W.M. is supported by an NHMRC Practitioner Fellowship (GNT1082548) and reports consultancy work for ObsEva, Merck, Merck KGaA, Guerbet and iGenomics. SB is Editor-in-Chief of Human Reproduction Open. None of the other authors declare any conflicts of interest.
Infertility is a significant health problem and assisted reproductive technologies to treat infertility. Despite all efforts, the success rate of these methods is still low. Also, each of these methods has side effects and costs. Therefore, accurate prediction of treatment success rate is a clinical challenge. This retrospective study aimed to internally validate and compare various machine learning models for predicting the clinical pregnancy rate (CPR) of infertility treatment. For this purpose, data from 1931 patients consisting of in vitro fertilization (IVF) or intra cytoplasmic sperm injection (ICSI) (733) and intra uterine insemination (IUI) (1196) treatments were included. Also, no egg or sperm donation data were used. The performance of machine learning algorithms to predict clinical pregnancy were expressed in terms of accuracy, recall, F-score, positive predictive value (PPV), brier score (BS), Matthew correlation coefficient (MCC), and receiver operating characteristic. The significance of the features with CPR and AUCs was evaluated by Student's t test and DeLong’s algorithm. Random forest (RF) model had the highest accuracy in the IVF/ICSI treatment. The sensitivity, F1 score, PPV, and MCC of the RF model were 0.76, 0.73, 0.80, and 0.5, respectively. These values for IUI treatment were 0.84, 0.80, 0.82, and 0.34, respectively. The BS was 0.13 and 0.15 for IVF/ICS and IUI, respectively. In addition, the estimated AUCs of the RF model for IVF/ICS and IUI were 0.73 and 0.7, respectively. Some essential features were obtained based on RF ranking for the two datasets, including age, follicle stimulation hormone, endometrial thickness, and infertility duration. The results showed a strong relationship between clinical pregnancy and a woman's age. Also, endometrial thickness and the number of follicles decreased with increasing female age in both treatments.
STUDY QUESTION Can we develop an IVF prediction model to estimate individualized chances of a live birth over multiple complete cycles of IVF in couples embarking on their second complete cycle of treatment? SUMMARY ANSWER Yes, our prediction model can estimate individualized chances of cumulative live birth over three additional complete cycles of IVF. WHAT IS KNOWN ALREADY After the completion of a first complete cycle of IVF, couples who are unsuccessful may choose to undergo further treatment to have their first child, while those who have had a live birth may decide to have more children. Existing prediction models can estimate the overall chances of success in couples before commencing IVF but are unable to revise these chances on the basis of the couple’s response to a first treatment cycle in terms of the number of eggs retrieved and pregnancy outcome. This makes it difficult for couples to plan and prepare emotionally and financially for the next step in their treatment. STUDY DESIGN, SIZE, DURATION For model development, a population-based cohort was used of 49 314 women who started their second cycle of IVF including ICSI in the UK from 1999 to 2008 using their own oocytes and their partners’ sperm. External validation was performed on data from 39 442 women who underwent their second cycle from 2010 to 2016. PARTICIPANTS/MATERIALS, SETTING, METHODS Data about all UK IVF treatments were obtained from the Human Fertilisation and Embryology Authority (HFEA) database. Using a discrete time logistic regression model, we predicted the cumulative probability of live birth from the second up to and including the fourth complete cycles of IVF. Inverse probability weighting was used to account for treatment discontinuation. Discrimination was assessed using c-statistic and calibration was assessed using calibration-in-the-large and calibration slope. MAIN RESULTS AND THE ROLE OF CHANCE Following exclusions, 49 314 women with 73 053 complete cycles were included. 12 408 (25.2%) had a live birth resulting from their second complete cycle. Cumulatively, 17 394 (35.3%) had a live birth over complete cycles two to four. The model showed moderate discriminative ability (c-statistic: 0.65, 95% CI: 0.64 to 0.65) and evidence of overprediction (calibration-in-the-large = −0.08) and overfitting (calibration slope 0.85, 95% CI: 0.81 to 0.88) in the validation cohort. However, after recalibration the fit was much improved. The recalibrated model identified the following key predictors of live birth: female age (38 versus 32 years—adjusted odds ratio: 0.59, 95% CI: 0.57 to 0.62), number of eggs retrieved in the first complete cycle (12 versus 4 eggs; 1.34, 1.30 to 1.37) and outcome of the first complete cycle (live birth versus no pregnancy; 1.78, 1.66 to 1.91; live birth versus pregnancy loss; 1.29, 1.23 to 1.36). As an example, a 32-year-old with 2 years of non-tubal infertility who had 12 eggs retrieved from her first stimulation and had a live birth during her first complete cycle has a 46% chance of having a further live birth from the second complete cycle of IVF and an 81% chance over a further three cycles. LIMITATIONS, REASONS FOR CAUTION The developed model was updated using validation data that was 6 to 12 years old. IVF practice continues to evolve over time, which may affect the accuracy of predictions from the model. We were unable to adjust for some potentially important predictors, e.g. BMI, smoking and alcohol intake in women, as well as measures of ovarian reserve such as antral follicle count. These were not available in the linked HFEA dataset. WIDER IMPLICATIONS OF THE FINDINGS By appropriately adjusting for couples who discontinue treatment, our novel prediction model will provide more realistic chances of live birth in couples starting a second complete cycle of IVF. Clinicians can use these predictions to inform discussion with couples who wish to plan ahead. This prediction tool will enable couples to prepare emotionally, financially and logistically for IVF treatment. STUDY FUNDING/COMPETING INTEREST(S) This study was supported by an Elphinstone scholarship scheme at the University of Aberdeen and Aberdeen Fertility Centre, University of Aberdeen. The authors have no conflict of interest. TRIAL REGISTRATION NUMBER N/A.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.