Clinical value of radiomics and machine learning in breast ultrasound: a multicenter study for differential diagnosis of benign and malignant lesions

Abstract

Objectives We aimed to assess the performance of radiomics and machine learning (ML) for classification of non-cystic benign and malignant breast lesions on ultrasound images, compare ML’s accuracy with that of a breast radiologist, and verify if the radiologist’s performance is improved by using ML. Methods Our retrospective study included patients from two institutions. A total of 135 lesions from Institution 1 were used to train and test the ML model with cross-validation. Radiomic features were extracted from manually annotated images and underwent a multistep feature selection process. Not reproducible, low variance, and highly intercorrelated features were removed from the dataset. Then, 66 lesions from Institution 2 were used as an external test set for ML and to assess the performance of a radiologist without and with the aid of ML, using McNemar’s test. Results After feature selection, 10 of the 520 features extracted were employed to train a random forest algorithm. Its accuracy in the training set was 82% (standard deviation, SD, ± 6%), with an AUC of 0.90 (SD ± 0.06), while the performance on the test set was 82% (95% confidence intervals (CI) = 70–90%) with an AUC of 0.82 (95% CI = 0.70–0.93). It resulted in being significantly better than the baseline reference (p = 0.0098), but not different from the radiologist (79.4%, p = 0.815). The radiologist’s performance improved when using ML (80.2%), but not significantly (p = 0.508). Conclusions A radiomic analysis combined with ML showed promising results to differentiate benign from malignant breast lesions on ultrasound images. Key Points • Machine learning showed good accuracy in discriminating benign from malignant breast lesions • The machine learning classifier’s performance was comparable to that of a breast radiologist • The radiologist’s accuracy improved with machine learning, but not significantly

Machine learning based on automated breast volume scanner (ABVS) radiomics for differential diagnosis of benign and malignant BI‐RADS 4 lesions

“…21 In our study, the selected ML pipeline (DNN-RFE) based on ABVS radiomics also showed satisfactory discrimination performance in the testing cohort, with the AUC, sensitivity, specificity, and accuracy of 0.954, 95.2%, 83.3%, and 88.9%, respectively. Interestingly, Romeo et al 30 used the ML algorithm (RF) based on US radiomics to distinguish benign and malignant breast lesions with an accuracy of 82% and an AUC of 0.82, which is lower than that of our study (88.9% and 0.954), although our study focuses on more challenging lesions (BI-RADS 4). The possible reason is that the three ABVS images (axial, sagittal, and coronal) may provide more radiomic features and better represent tumor heterogeneity than a single US Thus, ABVS images are indeed suitable for radiomics and ML methods, and may provide clinical decision support for the management and treatment of breast cancer.…”

Efficient Prediction of Ki-67 Proliferation Index in Meningiomas on MRI: From Traditional Radiological Findings to a Machine Learning Approach

“…One major concern should be noted that there were too many features involved in their modeling compared to ours (60 vs. 14). The most convincing method to identify whether the trained model is overfitted is externally testing it on unseen data obtained from another institution [ 29 , 30 ]. The results of our external test suggest there was a moderate overfitting in our models, even if only 14 features were used as classifier inputs.…”

Ultrasound radiomics in personalized breast management: Current status and future prospects

“…Although ultrasonography is the one of most common imaging technique used to detect and distinguish benign and malignant breast lesions, it is difficult to accurately and stably identify some lesions with the naked eye. Recently, many studies have explored the potential of ultrasound radiomics to aid in the detection and differentiation of lesions ( 9 , 17 – 20 ) ( Table 1 ).…”

“…This study confirmed that the DLR model had equal or better diagnostic performance compared to radiologists on a test dataset with 120 breast lesions (AUC = 0.913 vs 0.728-0.845, p = 0.01-0.14). Subsequently, several studies have shown that ultrasound radiomics based on 2D-US images has good performance in identifying benign from malignant breast lesion, with AUCs ranging from 0.817-0.943 ( 9 , 17 – 19 ). Additionally, studies have shown that the classification performance of the AI model may be affected by adjusting the ROI as different inputs of the model.…”

“…New opportunities have emerged with the advent of radiomics, a technique for extracting high-throughput quantitative features from medical images In recent years, radiomics based on X-ray, US, magnetic resonance imaging (MRI) and positron emission tomography-computed tomography (PET-CT) has proved to be useful for extracting a large number of image features that cannot be observed with the naked eye (7)(8)(9)(10). In some tasks, it matches or exceeds human perception (11,12).…”