Development and Validation of an Automated Classifier to Diagnose Acute Otitis Media in Children

Abstract

ImportanceAcute otitis media (AOM) is a frequently diagnosed illness in children, yet the accuracy of diagnosis has been consistently low. Multiple neural networks have been developed to recognize the presence of AOM with limited clinical application.ObjectiveTo develop and internally validate an artificial intelligence decision-support tool to interpret videos of the tympanic membrane and enhance accuracy in the diagnosis of AOM.Design, Setting, and ParticipantsThis diagnostic study analyzed otoscopic videos of the tympanic membrane captured using a smartphone during outpatient clinic visits at 2 sites in Pennsylvania between 2018 and 2023. Eligible participants included children who presented for sick visits or wellness visits.ExposureOtoscopic examination.Main Outcomes and MeasuresUsing the otoscopic videos that were annotated by validated otoscopists, a deep residual-recurrent neural network was trained to predict both features of the tympanic membrane and the diagnosis of AOM vs no AOM. The accuracy of this network was compared with a second network trained using a decision tree approach. A noise quality filter was also trained to prompt users that the video segment acquired may not be adequate for diagnostic purposes.ResultsUsing 1151 videos from 635 children (majority younger than 3 years of age), the deep residual-recurrent neural network had almost identical diagnostic accuracy as the decision tree network. The finalized deep residual-recurrent neural network algorithm classified tympanic membrane videos into AOM vs no AOM categories with a sensitivity of 93.8% (95% CI, 92.6%-95.0%) and specificity of 93.5% (95% CI, 92.8%-94.3%) and the decision tree model had a sensitivity of 93.7% (95% CI, 92.4%-94.9%) and specificity of 93.3% (92.5%-94.1%). Of the tympanic membrane features outputted, bulging of the TM most closely aligned with the predicted diagnosis; bulging was present in 230 of 230 cases (100%) in which the diagnosis was predicted to be AOM in the test set.Conclusions and RelevanceThese findings suggest that given its high accuracy, the algorithm and medical-grade application that facilitates image acquisition and quality filtering could reasonably be used in primary care or acute care settings to aid with automated diagnosis of AOM and decisions regarding treatment.

AI app that can diagnose acute otitis media may cut antibiotic use, say researchers

“…The team developed two AI models to analyse a few seconds of video taken from an otoscope connected to a phone camera to detect acute otitis media by assessing features of the eardrum, including its shape, position, colour, and translucency, they reported in JAMA Pediatrics 1…”

Seeking Details on Artificial Intelligence Diagnosis for Acute Otitis Media—Reply

“…In Reply We appreciate the interest and support of Walston et al for our use of an artificial intelligence decision-support tool to enhance accuracy of diagnosis for acute otitis media (AOM) in young children . They ask for further details on training, validation, and testing process of the deep learning models, specifically whether a 20-fold cross-validation was used and how data were partitioned.…”

Seeking Details on Artificial Intelligence Diagnosis for Acute Otitis Media

“…To the Editor We read with great interest the recent article by Shaikh et al The authors present a compelling study on the use of an artificial intelligence decision-support tool to enhance the accuracy of diagnosing acute otitis media in young children. We commend the authors on their innovative approach and noteworthy findings; however, we seek clarification on several methodological aspects that we believe are crucial for the robustness and reproducibility of the study.…”