Lymph Node Metastases in Papillary Thyroid Carcinoma can be Predicted by a Convolutional Neural Network: a Multi-Institution Study

Abstract

Objectives: The presence of nodal metastases in patients with papillary thyroid carcinoma (PTC) has both staging and treatment implications. However, lymph nodes are often not removed during thyroidectomy. Prior work has demonstrated the capability of artificial intelligence (AI) to predict the presence of nodal metastases in PTC based on the primary tumor histopathology alone. This study aimed to replicate these results with multi-institutional data. Methods: Cases of conventional PTC were identified from the records of 2 large academic institutions. Only patients with complete pathology data, including at least 3 sampled lymph nodes, were included in the study. Tumors were designated “positive” if they had at least 5 positive lymph node metastases. First, algorithms were trained separately on each institution’s data and tested independently on the other institution’s data. Then, the data sets were combined and new algorithms were developed and tested. The primary tumors were randomized into 2 groups, one to train the algorithm and another to test it. A low level of supervision was used to train the algorithm. Board-certified pathologists annotated the slides. HALO-AI convolutional neural network and image software was used to perform training and testing. Receiver operator characteristic curves and the Youden J statistic were used for primary analysis. Results: There were 420 cases used in analyses, 45% of which were negative. The best performing single institution algorithm had an area under the curve (AUC) of 0.64 with a sensitivity and specificity of 65% and 61% respectively, when tested on the other institution’s data. The best performing combined institution algorithm had an AUC of 0.84 with a sensitivity and specificity of 68% and 91% respectively. Conclusion: A convolutional neural network can produce an accurate and robust algorithm that is capable of predicting nodal metastases from primary PTC histopathology alone even in the setting of multi-institutional data.

Artificial intelligence in screening and diagnosis of surgical diseases: A narrative review

“…The best-performing institution's combined algorithm had a sensitivity and specificity of 68% and 91%, respectively. The authors concluded that the results were comparable to the ones obtained in their previous study, confirming the future potential advantage of using AI technologies to generate algorithms able to predict the presence of lymph node metastases in patients with PTC [87] .…”

“…Recently, the same authors conducted a follow-on multicenter study aimed at further assessing the ability of a CNN to predict the presence or absence of lymph node metastases in patients with PTC [87] . When CNN from one institution was tested against images from the other institution, the achieved sensitivity and specificity were 65% and 61%, respectively.…”

Trends in AI-powered classification of thyroid neoplasms based on histopathology images, a systematic review

“…A study by Anand et al 2021 described a weakly supervised technique to train a DNN (deep neural network) to predict BRAF V600E mutational status with high accuracy, in unannotated H&E-stained images of thyroid cancer tissue (12). Two studies by Esce et al 2021 & 2023 demonstrated the efficacy of CNN in predicting the probability of nodal metastases in PTC (papillary thyroid carcinoma) based on the histopathology features of the primary tumor (13, 14). Finally, a study by Nojima et al 2023 compared the prediction accuracy of 3 CNN models in distinguishing FA (follicular adenoma) from FTC (follicular thyroid carcinoma) based on histopathology images (15).…”

Trends in AI-powered Classification of Thyroid Neoplasms Based on Histopathology Images - a Systematic Review

“…These included a study by Anand et al 2021 ( 12 ) that described a weakly supervised technique to train a DNN (deep neural network) to predict BRAF V600E mutational status with high accuracy in unannotated H&E-stained images of thyroid cancer tissue. Two studies by Esce et al ( 13 , 14 ) demonstrated the efficacy of CNNs in predicting the probability of nodal metastases in PTC (papillary thyroid carcinoma) based on the histopathology features of the primary tumor. Finally, a study by Nojima et al 2023 ( 15 ) compared the prediction accuracy of 3 CNN models in distinguishing FA (follicular adenoma) from FTC (follicular thyroid carcinoma) based on histopathology images.…”