Generalisability of fetal ultrasound deep learning models to low-resource imaging settings in five African countries

Abstract

Most artificial intelligence (AI) research and innovations have concentrated in high-income countries, where imaging data, IT infrastructures and clinical expertise are plentiful. However, slower progress has been made in limited-resource environments where medical imaging is needed. For example, in Sub-Saharan Africa, the rate of perinatal mortality is very high due to limited access to antenatal screening. In these countries, AI models could be implemented to help clinicians acquire fetal ultrasound planes for the diagnosis of fetal abnormalities. So far, deep learning models have been proposed to identify standard fetal planes, but there is no evidence of their ability to generalise in centres with low resources, i.e. with limited access to high-end ultrasound equipment and ultrasound data. This work investigates for the first time different strategies to reduce the domain-shift effect arising from a fetal plane classification model trained on one clinical centre with high-resource settings and transferred to a new centre with low-resource settings. To that end, a classifier trained with 1792 patients from Spain is first evaluated on a new centre in Denmark in optimal conditions with 1008 patients and is later optimised to reach the same performance in five African centres (Egypt, Algeria, Uganda, Ghana and Malawi) with 25 patients each. The results show that a transfer learning approach for domain adaptation can be a solution to integrate small-size African samples with existing large-scale databases in developed countries. In particular, the model can be re-aligned and optimised to boost the performance on African populations by increasing the recall to $$0.92 \pm 0.04$$ 0.92 ± 0.04 and at the same time maintaining a high precision across centres. This framework shows promise for building new AI models generalisable across clinical centres with limited data acquired in challenging and heterogeneous conditions and calls for further research to develop new solutions for the usability of AI in countries with fewer resources and, consequently, in higher need of clinical support.

Validity evidence supporting clinical skills assessment by artificial intelligence compared with trained clinician raters

“…This has implications for the interpretation of our study results as we may understate the potential value of AI‐based assessments. The robustness of the AIBA model may be improved by including more diverse training sets and by adjusting the model's hyperparameters to avoid overfitting to the training dataset 46–49 . However, the need for cross‐validation and very large datasets may ultimately hinder the accessibility and use of AI for assessment purposes, in particular, when compared with EBA that work after minimal rater instruction.…”

Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology

“…AI algorithms for the automated detection of various standard planes in US video scans have been reported by Płotka et al [25], Chen et al [28], and Baumgartner et al [15]. The unique study of Sendra-Balcells et al presented a deep-learning model to identify standard planes in 2D images showing the transferability of the AI method to six low-income African countries [17]. In comparison with the analysis of 2D US videos, Sridar et al [26], Burgos-Artizzu et al [27], Rahman et al [30], and Carneiro et al [31,32] reported AI systems for the automated detection or measurement of various standard planes in 2D US images.…”

“…Equipment-related factors: Especially in hospital settings, repeated examinations may be performed with different US machines, resulting in heterogenous data. Furthermore, image quality depends on resource availability and access to high-end US devices [17], or the use of point-of-care devices [18]. • Patient-related factors: Maternal obesity is known to have an impact on image quality and visualization of the fetus, and thus limits the accuracy of obstetric US examinations [19].…”

“…Only three of the included studies investigated the use of AI algorithms in 3D US images [22][23][24], while 24 focused on 2D images. Most studies reported the combined analysis of various standard planes (13 studies, [15,17,[23][24][25][26][27][28][29][30][31][32][33]) or exclusively presented an algorithm for the analysis of HC (9 studies, [18,22,[34][35][36][37][38][39][40]), AC (4 studies, [21,[41][42][43]) or femur length (1 study, [44]). Five studies used the freely available 'HC18' data set [45] for training and testing the algorithms, which contained 1334 2D images from 551 women of the standard fetal head plane [18,34,37,38,40].…”

“…In conclusion, the automated acquisition and measurement of standard planes is an increasingly investigated area, but still faces problems when it comes to clinical applicability and generalization. The benefits of AI usage in standard plane acquisition were found to be the possibility of real-time applicability [15,25,32,33]; the incorporation of clinical aspects into image interpretation [21,23,26]; the feasibility of biometric assessment by non-experts [35,39,43]; the application of a lightweight algorithm in point-of-care devices [18,34,37]; and, as a consequence of the latter two aspects, applicability for medically underserved areas [17,35].…”