mr$$^2$$NST: Multi-resolution and Multi-reference Neural Style Transfer for Mammography

Wang, Sheng; Huo, Jiayu; Ouyang, Xin; Che, Jifei; Zhang, Xue; Shen, Dinggang; Wang, Qian; Cheng, Jie

doi:10.1007/978-3-030-59354-4_16

Cited by 5 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The technique manipulates the sequential representations across a CNN to transfer the style of one image to another while keeping the original content [ 1 ]. Gatys et al [ 62 ] first proposed NST, which typically takes two input images: a content image C to be transferred and a style reference image S. It executes feature learning from the representations of Fl(C) and Fl(S) in layer l of a neural style transfer network [ 63 ]. However, suppose the image styles from different datasets are way too far.…”

Section: Advanced Augmentation Techniquesmentioning

confidence: 99%

Image Augmentation Techniques for Mammogram Analysis

et al. 2022

View full text Add to dashboard Cite

Research in the medical imaging field using deep learning approaches has become progressively contingent. Scientific findings reveal that supervised deep learning methods’ performance heavily depends on training set size, which expert radiologists must manually annotate. The latter is quite a tiring and time-consuming task. Therefore, most of the freely accessible biomedical image datasets are small-sized. Furthermore, it is challenging to have big-sized medical image datasets due to privacy and legal issues. Consequently, not a small number of supervised deep learning models are prone to overfitting and cannot produce generalized output. One of the most popular methods to mitigate the issue above goes under the name of data augmentation. This technique helps increase training set size by utilizing various transformations and has been publicized to improve the model performance when tested on new data. This article surveyed different data augmentation techniques employed on mammogram images. The article aims to provide insights into basic and deep learning-based augmentation techniques.

show abstract

Section: Advanced Augmentation Techniquesmentioning

confidence: 99%

Image Augmentation Techniques for Mammogram Analysis

et al. 2022

View full text Add to dashboard Cite

show abstract

“…There are several recent studies which include or relate to accommodating for subtle image differences. Recent approaches include an encoder-decoder module at the top of a CNN (18), batch-instance normalisation (18) and neural style transfer (33). This may be significant in the event that a screening service changes equipment vendors, requiring transition of an existing model to slightly different images.…”

Section: Work By Wu Et Almentioning

confidence: 99%

Replication of an open-access deep learning system for screening mammography: Reduced performance mitigated by retraining on local data

Jjj

Oakden-Rayner

et al. 2021

Preprint

View full text Add to dashboard Cite

Aim To assess the generalisability of a deep learning (DL) system for screening mammography developed at New York University (NYU), USA (1,2) in a South Australian (SA) dataset. Methods and Materials Clients with pathology-proven lesions (n=3,160) and age-matched controls (n=3,240) were selected from women screened at BreastScreen SA from January 2010 to December 2016 (n clients=207,691) and split into training, validation and test subsets (70\%, 15\%, 15\% respectively). The primary outcome was area under the curve (AUC), in the SA Test Set 1 (SATS1), differentiating invasive breast cancer or ductal carcinoma in situ (n=469) from age-matched controls (n=490) and benign lesions (n=44). The NYU system was tested statically, after training without transfer learning (TL), after retraining with TL and without (NYU1) and with (NYU2) heatmaps. Results The static NYU1 model AUCs in the NYU test set (NYTS) and SATS1 were 83.0\%(95\%CI=82.4\%-83.6\%)(2) and 75.8\%(95\%CI=72.6\%-78.8\%), respectively. Static NYU2 AUCs in the NYTS and SATS1 were 88.6\%(95\%CI=88.3\%-88.9\%)(2) and 84.5\%(95\%CI=81.9\%-86.8\%), respectively. Training of NYU1 and NYU2 without TL achieved AUCs in the SATS1 of 65.8\% (95\%CI=62.2\%-69.1\%) and 85.9\%(95\%CI=83.5\%-88.2\%), respectively. Retraining of NYU1 and NYU2 with TL resulted in AUCs of 82.4\%(95\%CI=79.7-84.9\%) and 86.3\%(95\%CI=84.0-88.5\%) respectively. Conclusion We did not fully reproduce the reported performance of NYU on a local dataset; local retraining with TL approximated this level of performance. Optimising models for local clinical environments may improve performance. The generalisation of DL systems to new environments may be challenging.

show abstract

“…Specifically, given M seen vendor-style domain, we train M 2 generators, which map the data distribution of source domain Ω i to target Ω j domain, ∀i, j ∈ M . Comparing to the method used in [19], the CycleGAN realizes the style transfer with bidirectional learning process. The work [19] unidirectionally takes a few references images may attain limited transferring effect.…”

Section: Multi-style and Multi-view Contrastive Learningmentioning

confidence: 99%

“…Comparing to the method used in [19], the CycleGAN realizes the style transfer with bidirectional learning process. The work [19] unidirectionally takes a few references images may attain limited transferring effect.…”

Section: Multi-style and Multi-view Contrastive Learningmentioning

confidence: 99%

“…In the literature, the domain generalization for DL technique can be classified into three categories: 1) conventional data argumentation methods, e.g. rotation, flip, deformation and jittering [23,14,18], 2) learning based methods with the generative deep neural networks [21,19,6,22], 3) learning based methods for the exploration of task-specific and domain-invariant features [20,9,5,7,8,4,2]. However, all these learning based methods relied on annotated data, which are costly obtain and availability is relatively limited.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learning

Cui

Wang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Lesion detection is a fundamental problem in the computeraided diagnosis scheme for mammography. The advance of deep learning techniques have made a remarkable progress for this task, provided that the training data are large and sufficiently diverse in terms of image style and quality. In particular, the diversity of image style may be majorly attributed to the vendor factor. However, the collection of mammograms from vendors as many as possible is very expensive and sometimes impractical for laboratory-scale studies. Accordingly, to further augment the generalization capability of deep learning model to various vendors with limited resources, a new contrastive learning scheme is developed. Specifically, the backbone network is firstly trained with a multi-style and multi-view unsupervised self-learning scheme for the embedding of invariant features to various vendor-styles. Afterward, the backbone network is then recalibrated to the downstream task of lesion detection with the specific supervised learning. The proposed method is evaluated with mammograms from four vendors and one unseen public dataset. The experimental results suggest that our approach can effectively improve detection performance on both seen and unseen domains, and outperforms many state-of-the-art (SOTA) generalization methods.

show abstract

mr$$^2$$NST: Multi-resolution and Multi-reference Neural Style Transfer for Mammography

Cited by 5 publications

References 9 publications

Image Augmentation Techniques for Mammogram Analysis

Image Augmentation Techniques for Mammogram Analysis

Replication of an open-access deep learning system for screening mammography: Reduced performance mitigated by retraining on local data

Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learning

Contact Info

Product

Resources

About