Description and clinical validation of a real-time AI diagnostic companion for fetal ultrasound examination

Abstract

Objective: To describe a real-time decision support system (DSS), named SONIO, to assist ultrasound-based prenatal diagnosis and to assess its performance using a clinical database of precisely phenotyped postmortem examinations. Population and Methods: This DSS is knowledge-based and comprises a dedicated thesaurus of 294 syndromes and diseases. It operates by suggesting, at each step of the ultrasound examination, the best next symptom to check for in order to optimize the diagnostic pathway to the smallest number of possible diagnoses. This assistant was tested on a single-center database of 251 cases of postmortem phenotypes with a definite diagnosis. Adjudication of discordant diagnoses was made by a panel of external experts. The primary outcome was a target concordance rate >90% between the postmortem diagnosis and the top-7 diagnoses given by SONIO when providing the full phenotype as input. Secondary outcomes included concordance for the top-5 and top-3 diagnoses; We also assessed a '1-by-1' model, providing only the anomalies sequentially prompted by the system, mimicking the use of the software in a real-life clinical setting. Results: The validation database covered 96 of the 294 (32.65%) syndromes and 79% of their overall prevalence in the SONIO thesaurus. The adjudicators discarded 42/251 cases as they were not amenable to ultrasound based diagnosis. SONIO failed to make the diagnosis on 7/209 cases. On average, each case displayed 6 anomalies, 3 of which were considered atypical for the condition. Using the 'full-phenotype' model, the success rate of the top-7 output of Sonio was 96.7% (202/209). This was 91.9% and 87.1% for the top-5 and top-3 outputs respectively. Using the '1-by-1' model, the correct diagnosis was within the top-7, top-5 and top-3 of SONIO's output in 72.4%, 69.3% and 63.1%. Conclusion: Sonio is a robust DSS with a success-rate >95% for top-7 ranking diagnoses when the full phenotype is provided, using a large database of noisy real data. The success rate over 70% using the '1-by-1' model was understandably lower, given that SONIO's sequential queries may not systematically cover the full phenotype.