Ensemble neural network model for detecting thyroid eye disease using external photographs

Abstract

PurposeTo describe an artificial intelligence platform that detects thyroid eye disease (TED).DesignDevelopment of a deep learning model.Methods1944 photographs from a clinical database were used to train a deep learning model. 344 additional images (‘test set’) were used to calculate performance metrics. Receiver operating characteristic, precision–recall curves and heatmaps were generated. From the test set, 50 images were randomly selected (‘survey set’) and used to compare model performance with ophthalmologist performance. 222 images obtained from a separate clinical database were used to assess model recall and to quantitate model performance with respect to disease stage and grade.ResultsThe model achieved test set accuracy of 89.2%, specificity 86.9%, recall 93.4%, precision 79.7% and an F1 score of 86.0%. Heatmaps demonstrated that the model identified pixels corresponding to clinical features of TED. On the survey set, the ensemble model achieved accuracy, specificity, recall, precision and F1 score of 86%, 84%, 89%, 77% and 82%, respectively. 27 ophthalmologists achieved mean performance of 75%, 82%, 63%, 72% and 66%, respectively. On the second test set, the model achieved recall of 91.9%, with higher recall for moderate to severe (98.2%, n=55) and active disease (98.3%, n=60), as compared with mild (86.8%, n=68) or stable disease (85.7%, n=63).ConclusionsThe deep learning classifier is a novel approach to identify TED and is a first step in the development of tools to improve diagnostic accuracy and lower barriers to specialist evaluation.

Machine learning-assisted system using digital facial images to predict the clinical activity score in thyroid-associated orbitopathy

“…This study has strengths that the proposed system can detect the early stage of TAO based on CAS before serious changes in appearance occur. Recently, two studies to diagnose TAO from digital facial images using AI models were published 20,21 . Karlin et al 20 reported a deep learning-based classifier to identify the presence of TAO, not the active phase of TAO which can be derived from CAS, based on facial images where the classifier is the ensemble of ResNet 18 networks.…”

“…Recently, two studies to diagnose TAO from digital facial images using AI models were published 20,21 . Karlin et al 20 reported a deep learning-based classifier to identify the presence of TAO, not the active phase of TAO which can be derived from CAS, based on facial images where the classifier is the ensemble of ResNet 18 networks. Using the major feature of TAO that the disease may result in facial disfigurement, they could achieve accuracy of 89.2% (sensitivity 93.4%; specificity 86.9%).…”

Research progress and application of artificial intelligence in thyroid associated ophthalmopathy

“… Karlin et al (2022) developed another AI platform based on a DL model to identify TAO using ocular photographs. The training set contained 1944 facial images, and the testing depended on additional 344 photographs.…”

“…The physiognomic changes in patients can be crucial for a real-time evaluation of the disease stage in the clinical diagnosis and management of TAO. The storage of facial images is important, which can also be used in AI training as mentioned earlier ( Huang et al, 2022 ; Karlin et al, 2022 ). The facial privacy of patients was commonly anonymized by cropping images into a restricted area in the overwhelming majority of data collection and literature reports.…”

Orbital and eyelid diseases: The next breakthrough in artificial intelligence?

“…Huang et al (2022) used the ResNet-50 model to obtain an automatic diagnosis of TAO based on external ocular photographs. Karlin J et al (2022) developed a DL model for detecting TAO based on external ocular photographs. A set comprising 1944 photographs from a clinical database was used for training, and a test set of 344 additional images was used to evaluate the trained DL network.…”