HoVer-Trans: Anatomy-Aware HoVer-Transformer for ROI-Free Breast Cancer Diagnosis in Ultrasound Images

Mo, Yuhao; Han, Chu; Liu, Yu; Liu, Min; Shi, Zhenwei; Lin, Jiatai; Zhao, Bingchao; Huang, Chunwang; Qiu, Bingjiang; Cui, Yanfen; Wu, Lei; Pan, Xipeng; Xu, Zeyan; Huang, Xiaomei; Li, Zhenhui; Liu, Zaiyi; Wang, Ying; Chen, Liang

doi:10.1109/tmi.2023.3236011

Cited by 28 publications

(7 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to its wide-ranging applications and outstanding performance, ResNet50 has become a benchmark model for many computer vision tasks, finding extensive use in areas such as object detection, image segmentation, and image generation. It has shown remarkable results in the evaluation of breast cancer, gastric cancer, spinal metastasis, and other tumors (Mo et al 2023 ; Iwaya et al 2023 ; Liu et al 2023 ). Furthermore, the features we extract are not task specific and are not dependent on a single task.…”

Section: Discussionmentioning

confidence: 99%

Deep learning algorithm-based multimodal MRI radiomics and pathomics data improve prediction of bone metastases in primary prostate cancer

Zhang,

Zhou,

Guo

et al. 2024

J Cancer Res Clin Oncol

View full text Add to dashboard Cite

Purpose Bone metastasis is a significant contributor to morbidity and mortality in advanced prostate cancer, and early diagnosis is challenging due to its insidious onset. The use of machine learning to obtain prognostic information from pathological images has been highlighted. However, there is a limited understanding of the potential of early prediction of bone metastasis through the feature combination method from various sources. This study presents a method of integrating multimodal data to enhance the feasibility of early diagnosis of bone metastasis in prostate cancer. Methods and materials Overall, 211 patients diagnosed with prostate cancer (PCa) at Gansu Provincial Hospital between January 2017 and February 2023 were included in this study. The patients were randomized (8:2) into a training group (n = 169) and a validation group (n = 42). The region of interest (ROI) were segmented from the three magnetic resonance imaging (MRI) sequences (T2WI, DWI, and ADC), and pathological features were extracted from tissue sections (hematoxylin and eosin [H&E] staining, 10 × 20). A deep learning (DL) model using ResNet 50 was employed to extract deep transfer learning (DTL) features. The least absolute shrinkage and selection operator (LASSO) regression method was utilized for feature selection, feature construction, and reducing feature dimensions. Different machine learning classifiers were used to build predictive models. The performance of the models was evaluated using receiver operating characteristic curves. The net clinical benefit was assessed using decision curve analysis (DCA). The goodness of fit was evaluated using calibration curves. A joint model nomogram was eventually developed by combining clinically independent risk factors. Results The best prediction models based on DTL and pathomics features showed area under the curve (AUC) values of 0.89 (95% confidence interval [CI], 0.799–0.989) and 0.85 (95% CI, 0.714–0.989), respectively. The AUC for the best prediction model based on radiomics features and combining radiomics features, DTL features, and pathomics features were 0.86 (95% CI, 0.735–0.979) and 0.93 (95% CI, 0.854–1.000), respectively. Based on DCA and calibration curves, the model demonstrated good net clinical benefit and fit. Conclusion Multimodal radiomics and pathomics serve as valuable predictors of the risk of bone metastases in patients with primary PCa.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep learning algorithm-based multimodal MRI radiomics and pathomics data improve prediction of bone metastases in primary prostate cancer

Zhang,

Zhou,

Guo

et al. 2024

J Cancer Res Clin Oncol

View full text Add to dashboard Cite

show abstract

“…We have enhanced the dataset and expanded each ultrasound image to 20 and 500 BUS datasets to 10,000 using rotation, mirroring, brightness change, Gaussian noise, and other data enhancement technologies. Breast ultrasonography is the use of ultrasonic physical signals to diagnose breast diseases; ultrasound is delivered through the probe in the human breast to reach the surface of various tissues and organs and produces echo signals, collecting strong and weak signals and long and short echo times, thus forming the structure of human breast tissue image examination ( 16 ). Typical breast cancer node B ultrasound images are shown in Figure 1 .…”

Section: Methodsmentioning

confidence: 99%

“…Mo et al. first predefined the regions of interest (ROIs) and then classified the lesion inside the ROIs; then, they used the so-called HoVer-Trans block to extract the inter- and intralayer spatial information horizontally and vertically ( 16 ). With the development of immunohistochemical technology, it becomes more and more involved in cancer classifications.…”

Section: Introductionmentioning

confidence: 99%

Deep learning-based immunohistochemical estimation of breast cancer via ultrasound image applications

Yan,

Zhao,

Duan

et al. 2024

Front. Oncol.

View full text Add to dashboard Cite

BackgroundBreast cancer is the key global menace to women’s health, which ranks first by mortality rate. The rate reduction and early diagnostics of breast cancer are the mainstream of medical research. Immunohistochemical examination is the most important link in the process of breast cancer treatment, and its results directly affect physicians’ decision-making on follow-up medical treatment.PurposeThis study aims to develop a computer-aided diagnosis (CAD) method based on deep learning to classify breast ultrasound (BUS) images according to immunohistochemical results.MethodsA new depth learning framework guided by BUS image data analysis was proposed for the classification of breast cancer nodes in BUS images. The proposed CAD classification network mainly comprised three innovation points. First, a multilevel feature distillation network (MFD-Net) based on CNN, which could extract feature layers of different scales, was designed. Then, the image features extracted at different depths were fused to achieve multilevel feature distillation using depth separable convolution and reverse depth separable convolution to increase convolution depths. Finally, a new attention module containing two independent submodules, the channel attention module (CAM) and the spatial attention module (SAM), was introduced to improve the model classification ability in channel and space.ResultsA total of 500 axial BUS images were retrieved from 294 patients who underwent BUS examination, and these images were detected and cropped, resulting in breast cancer node BUS image datasets, which were classified according to immunohistochemical findings, and the datasets were randomly subdivided into a training set (70%) and a test set (30%) in the classification process, with the results of the four immune indices output simultaneously from training and testing, in the model comparison experiment. Taking ER immune indicators as an example, the proposed model achieved a precision of 0.8933, a recall of 0.7563, an F1 score of 0.8191, and an accuracy of 0.8386, significantly outperforming the other models. The results of the designed ablation experiment also showed that the proposed multistage characteristic distillation structure and attention module were key in improving the accuracy rate.ConclusionThe extensive experiments verify the high efficiency of the proposed method. It is considered the first classification of breast cancer by immunohistochemical results in breast cancer image processing, and it provides an effective aid for postoperative breast cancer treatment, greatly reduces the difficulty of diagnosis for doctors, and improves work efficiency.

show abstract

“…We also compared our hybrid method with four other relative state-of-the-art classification methods, including two deep learning methods for breast ultrasound and two hybrid methods for other domains. HoVer-Trans [38] was proposed based on vision transformer for breast cancer diagnosis in ultrasound images, according to the original study we resized images to 256 × 256 for model training. Another deep learning method for breast ultrasound is SBANet [17], which consisted of three training stage.…”

Section: Performance Evaluation With Comparison Methodsmentioning

confidence: 99%

Spatial and geometric learning for classification of breast tumors from multi-center ultrasound images: a hybrid learning approach

Ru,

Zhu,

Shi

2024

BMC Med Imaging

View full text Add to dashboard Cite

Background Breast cancer is the most common cancer among women, and ultrasound is a usual tool for early screening. Nowadays, deep learning technique is applied as an auxiliary tool to provide the predictive results for doctors to decide whether to make further examinations or treatments. This study aimed to develop a hybrid learning approach for breast ultrasound classification by extracting more potential features from local and multi-center ultrasound data. Methods We proposed a hybrid learning approach to classify the breast tumors into benign and malignant. Three multi-center datasets (BUSI, BUS, OASBUD) were used to pretrain a model by federated learning, then every dataset was fine-tuned at local. The proposed model consisted of a convolutional neural network (CNN) and a graph neural network (GNN), aiming to extract features from images at a spatial level and from graphs at a geometric level. The input images are small-sized and free from pixel-level labels, and the input graphs are generated automatically in an unsupervised manner, which saves the costs of labor and memory space. Results The classification AUCROC of our proposed method is 0.911, 0.871 and 0.767 for BUSI, BUS and OASBUD. The balanced accuracy is 87.6%, 85.2% and 61.4% respectively. The results show that our method outperforms conventional methods. Conclusions Our hybrid approach can learn the inter-feature among multi-center data and the intra-feature of local data. It shows potential in aiding doctors for breast tumor classification in ultrasound at an early stage.

show abstract

HoVer-Trans: Anatomy-Aware HoVer-Transformer for ROI-Free Breast Cancer Diagnosis in Ultrasound Images

Cited by 28 publications

References 38 publications

Deep learning algorithm-based multimodal MRI radiomics and pathomics data improve prediction of bone metastases in primary prostate cancer

Deep learning algorithm-based multimodal MRI radiomics and pathomics data improve prediction of bone metastases in primary prostate cancer

Deep learning-based immunohistochemical estimation of breast cancer via ultrasound image applications

Spatial and geometric learning for classification of breast tumors from multi-center ultrasound images: a hybrid learning approach

Contact Info

Product

Resources

About