Assessment of Axillary Lymph Nodes for Metastasis on Ultrasound Using Artificial Intelligence

Abstract

The purpose of this study was to evaluate an artificial intelligence (AI) system for the classification of axillary lymph nodes on ultrasound compared to radiologists. Ultrasound images of 317 axillary lymph nodes from patients referred for ultrasound guided fine needle aspiration or core needle biopsy and corresponding pathology findings were collected. Lymph nodes were classified into benign and malignant groups with histopathological result serving as the reference. Google Cloud AutoML Vision (Mountain View, CA) was used for AI image classification. Three experienced radiologists also classified the images and gave a level of suspicion score (1–5). To test the accuracy of AI, an external testing dataset of 64 images from 64 independent patients was evaluated by three AI models and the three readers. The diagnostic performance of AI and the humans were then quantified using receiver operating characteristics curves. In the complete set of 317 images, AutoML achieved a sensitivity of 77.1%, positive predictive value (PPV) of 77.1%, and an area under the precision recall curve of 0.78, while the three radiologists showed a sensitivity of 87.8% ± 8.5%, specificity of 50.3% ± 16.4%, PPV of 61.1% ± 5.4%, negative predictive value (NPV) of 84.1% ± 6.6%, and accuracy of 67.7% ± 5.7%. In the three external independent test sets, AI and human readers achieved sensitivity of 74.0% ± 0.14% versus 89.9% ± 0.06% ( p = .25), specificity of 64.4% ± 0.11% versus 50.1 ± 0.20% ( p = .22), PPV of 68.3% ± 0.04% versus 65.4 ± 0.07% ( p = .50), NPV of 72.6% ± 0.11% versus 82.1% ± 0.08% ( p = .33), and accuracy of 69.5% ± 0.06% versus 70.1% ± 0.07% ( p = .90), respectively. These preliminary results indicate AI has comparable performance to trained radiologists and could be used to predict the presence of metastasis in ultrasound images of axillary lymph nodes.

Ultrasound Elastography: Basic Principles and Examples of Clinical Applications with Artificial Intelligence—A Review

“…In normal LNs, the cortex is usually stiffer than the hilum, and this architecture is generally also preserved in inflammatory LNs. Conversely, malignant carcinoma cells proliferate rapidly and infiltrate the lymph node, distorting the normal architecture and increasing its stiffness [85]. However, focal infiltration may be challenging to detect, and lymphoma may produce soft lymph nodes that can predominantly have similar elasticity to the surrounding tissue, thus representing a specific challenge for this application [7].…”

“…A study by Tahmasebi et al assessed the accuracy of image classification software (Google Cloud AutoML Vision, Mountain View, CA) compared to three expert radiologists, regarding a dataset containing ultrasound images of 317 axillary lymph nodes, using the histopathology as a reference standard. They showed that AI has comparable performance to expert radiologists and could be used to predict the presence of metastases in ultrasound images of the axillary lymph nodes [85].…”

The Role of AI in Breast Cancer Lymph Node Classification: A Comprehensive Review

“…One study compared the performances of Google Cloud AutoML Vision (Mountain View, CA, USA) with the performances of three experienced radiologists [ 38 ]. Ultrasound images of 317 axillary lymph nodes from breast cancer patients were collected.…”

“…Ultrasound images of 317 axillary lymph nodes from breast cancer patients were collected. When evaluated on three independent test sets, the AI system achieved better specificity (64.4% vs. 50.1%) and positive predictive value (68.3% vs. 65.4%), while it achieved worse sensitivity (74.0% vs. 89.9%) and comparable accuracy (69.5% vs. 70.1%) ( Table 2 ) [ 38 ]. Hence, this study showed how an automated AI system yields performances similar to that of experienced radiologists.…”

Ultrasound‐Based Machine Learning Approach for Detection of Nonalcoholic Fatty Liver Disease

“…21,22 Our group investigated the ultrasound-based AI performance in thyroid lesions, lymphatic involvement in breast cancer, and retrospectively in NAFLD. [23][24][25][26] However, relatively few papers have prospectively investigated the performance of AI techniques in assessing liver fat content on using ultrasound. [27][28][29][30] It is hoped that AI can assist physicians in making more accurate and reproducible imaging diagnoses and also reduce physician's workload.…”