Comparative Analysis of AI-SONICTM Thyroid System and Six Thyroid Risk Stratification Guidelines in Papillary Thyroid Cancer: A Retrospective Cohort Study

Abstract

Aim The study aimed to compare the diagnostic performance of AI-SONICTM Thyroid System (AI-SONICTM) with six thyroid nodule ultrasound risk stratification systems, as well as the interobserver agreement among different-year ultrasound examiners using the same diagnostic approach. Methods This retrospective study included patients who underwent thyroid ultrasound examination and surgery between 2010 and 2022. Three ultrasound examiners with 2, 5, and 10 years of experience, respectively, used AI-SONICTM and six guidelines to risk-stratify the nodules. The diagnostic performance and interobserver agreement were assessed. Results A total of 370 thyroid nodules were included, including 195 papillary thyroid carcinomas (PTC) and 175 benign nodules. For physicians of varying seniority from low to high, AI-SONICTM had a moderate sensitivities of 82.56%, 83.08%, 84.62%, respectively, while AACE/ACE/AME had the highest diagnostic sensitivities (96.41%, 95.38%, 96.41%, respectively); And relatively higher specificities were 85.14%, 85.71%, 85.71% for KSThR, while moderate specificities with values of 84.0%, 85.14%, and 85.71%, respectively were found for AI-SONICTM; The accuracy was highest for ATA (excluding non-classifiable nodules), with values of 87.26%, 87.93%, and 88.82%, respectively, while the accuracy for AI-SONICTM were 83.24%, 84.05%, and 85.14%, respectively. The Kendall’s tau coefficient indicated strong or moderate interobserver agreement among all examiners using different diagnostic methods (Kendall’s tau coefficient >0.6, P<0.001). AI-SONICTM showed the highest interobserver agreement (Kendall’s tau coefficient=0.995, P<0.001). A binary probit regression analysis showed that nodules with cystic components had a significantly higher regression coefficient value of 0.983 (P=0.002), indicating that AI-SONICTM may have higher accuracy for nodules with cystic components. Conclusion AI-SONICTM and the six thyroid nodule ultrasound risk stratification systems showed high diagnostic performance for papillary thyroid carcinoma. All examiners showed strong or moderate interobserver agreement when using different diagnostic methods. AI-SONICTM may have higher accuracy for nodules with cystic components.