Automatic Calcification Morphology and Distribution Classification for Breast Mammograms With Multi-Task Graph Convolutional Neural Network

Abstract

The morphology and distribution of microcalcifications are the most important descriptors for radiologists to diagnose breast cancer based on mammograms. However, it is very challenging and time-consuming for radiologists to characterize these descriptors manually, and there also lacks of effective and automatic solutions for this problem. We observed that the distribution and morphology descriptors are determined by the radiologists based on the spatial and visual relationships among calcifications. Thus, we hypothesize that this information can be effectively modelled by learning a relationship-aware representation using graph convolutional networks (GCNs). In this study, we propose a multi-task deep GCN method for automatic characterization of both the morphology and distribution of microcalcifications in mammograms. Our proposed method transforms morphology and distribution characterization into node and graph classification problem and learns the representations concurrently. We trained and validated the proposed method in an in-house dataset and public DDSM dataset with 195 and 583 cases,respectively. The proposed method reaches good and stable results with distribution AUC at 0.812±0.043 and 0.873±0.019, morphology AUC at 0.663±0.016 and 0.700±0.044 for both in-house and public datasets. In both datasets, our proposed method demonstrates statistically significant improvements compared to the baseline models. The performance improvements brought by our proposed multi-task mechanism can be attributed to the association between the distribution and morphology of calcifications in mammograms, which is interpretable using graphical visualizations and consistent with the definitions of descriptors in the standard BI-RADS guideline. In short, we explore, for the first time, the

Frozen Large-scale Pretrained Vision-Language Models are an Effective Foundational Backbone for Enhancing Multimodal Breast Cancer Risk Assessment

“…Mammogram-based breast cancer studies can also be categorized based on their utilization of input data into several groups: those focusing on unimodal data, which may encompass whole mammograms [19]- [22], [25], [28], [29], [32], [33], [33], [35], [37]- [39], [41], [44], [48], [52]- [56] or lesion crops [18], [27], [34], [36], [40], [51], [57]; others concentrating on multi-view mammograms [17], [30], [50], [58], [58]- [63]; some addressing both whole mammograms and lesion crops [26], [31], [42]; and additional investigations involving the integration of multiple modalities, such as the combination of mammograms with ultrasounds or electronic health records (EHRs) [36], [49], [54], [64]- [72]. In this work, we focus on two modalities: mammogram-based lesion crops and EHR data containing radiographic features and clinical patient metadata for breast cancer prediction.…”

Spatial and geometric learning for classification of breast tumors from multi-center ultrasound images: a hybrid learning approach

“…The combination of these two networks has also been explored by researchers. In [24], the authors extracted features by CNN to construct graphs and then used GNN for automatic characterization of both the morphology and distribution of microcalcifications in mammograms. Researchers in [25] used a CNN to extract features from DCE-MRI scans and an autoencoder to represent genomic variant results or micro array expression features in a condensed latent space.…”