Consistency and objectivity of automated embryo assessments using deep neural networks

Abstract

Objective: To evaluate the consistency and objectivity of deep neural networks in embryo scoring and making disposition decisions for biopsy and cryopreservation in comparison to grading by highly trained embryologists. Design: Prospective double-blind study using retrospective data. Setting: U.S.-based large academic fertility center. Patients: Not applicable. Intervention(s): Embryo images (748 recorded at 70 hours postinsemination [hpi]) and 742 at 113 hpi) were used to evaluate embryologists and neural networks in embryo grading. The performance of 10 embryologists and a neural network were also evaluated in disposition decision making using 56 embryos. Main Outcome Measures: Coefficients of variation (%CV) and measures of consistencies were compared. Results: Embryologists exhibited a high degree of variability (%CV averages: 82.84% for 70 hpi and 44.98% for 113 hpi) in grading embryo. When selecting blastocysts for biopsy or cryopreservation, embryologists had an average consistency of 52.14% and 57.68%, respectively. The neural network outperformed the embryologists in selecting blastocysts for biopsy and cryopreservation with a consistency of 83.92%. Cronbach’s α analysis revealed an α coefficient of 0.60 for the embryologists and 1.00 for the network. Conclusions: The results of our study show a high degree of interembryologist and intraembryologist variability in scoring embryos, likely due to the subjective nature of traditional morphology grading. This may ultimately lead to less precise disposition decisions and discarding of viable embryos. The application of a deep neural network, as shown in our study, can introduce improved reliability and high consistency during the process of embryo selection and disposition, potentially improving outcomes in an embryology laboratory.

Performance of a deep learning based neural network in the selection of human blastocysts for implantation

“…Deep neural networks hold value in aiding clinical decision making and have received significant attention from the IVF community. The deep-neural network-based approach showcased here is an objective approach to one of the more subjective but important parts of a clinical IVF process-embryo selections for transfer ( Bormann et al, 2020 ). Since over 80% of fertility clinics rely on non-time lapse imaging systems as part of their clinical processes, such neural network-based algorithms that rely purely on static single timepoint images can effectively assist in decision making ( Dolinko et al, 2017 ).…”

“…Recently, machine learning has been proposed as a solution for automated analysis of embryo morphologies (21)(22)(23)(24)(25)(26). This work makes use of a deep convolutional neural network (CNN), a representation learning technique, that has been proven to be effective in image classification tasks.…”

“…In the past decade, advances in hardware compute performance and machine learning techniques have significantly improved their applicability in real-world medical and non-medical problems. Recently, machine learning has been proposed as a solution for automated analysis of embryo morphologies ( Rocha et al, 2017b ; Bormann et al, 2020 ; Dimitriadis et al, 2019 ; Thirumalaraju et al, 2019 ; Khosravi et al, 2019 ; Kanakasabapathy et al, 2019a ). This work makes use of a deep convolutional neural network (CNN), a representation learning technique, that has been proven to be effective in image classification tasks.…”

A SWOT Analysis of Human- and Machine Learning- Based Embryo Assessment

“…Bias may occur because of the wide variation in experience and skill levels of the embryologists [72]. Equally important, the variability refers to the inconsistent evaluation at a different time on the same embryo [73]. The diversity of human performance would affect the success of IVF treatment.…”

“…Research showed that a well-designed ML model outperformed experts in the prediction of the potential implantation of embryos [74]. The advancement of research enabled the development of MLbased applications that combine operator-independent procedure and high-processivity capabilities [73]. An ML model developed with a sufficient retrospective dataset for training a model and prognostic data features can provide reliable implantation prediction [74].…”