SpotTune: Transfer Learning Through Adaptive Fine-Tuning

Guo, Yunhui; Shi, Honghui; Kumar, Abhishek; Grauman, Kristen; Rosing, Tajana; Feris, Rogério

doi:10.1109/cvpr.2019.00494

Cited by 362 publications

(289 citation statements)

References 42 publications

Supporting

Mentioning

281

Contrasting

Unclassified

Order By: Relevance

“…In order to improve performance further, we apply fine-tuning and SpotTune, 16 both with and without HM. Fine-tuning on the limited number of regular MIP images does not show any improvement in performance; however, SpotTune leads to a ROC AUC of 0.825.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis

Singh¹,

Matthews²,

Shah³

et al. 2020

Medical Imaging 2020: Computer-Aided Diagnosis

View full text Add to dashboard Cite

Mammography-based screening has helped reduce the breast cancer mortality rate, but has also been associated with potential harms due to low specificity, leading to unnecessary exams or procedures, and low sensitivity. Digital breast tomosynthesis (DBT) improves on conventional mammography by increasing both sensitivity and specificity and is becoming common in clinical settings. However, deep learning (DL) models have been developed mainly on conventional 2D full-field digital mammography (FFDM) or scanned film images. Due to a lack of large annotated DBT datasets, it is difficult to train a model on DBT from scratch. In this work, we present methods to generalize a model trained on FFDM images to DBT images. In particular, we use average histogram matching (HM) and DL fine-tuning methods to generalize a FFDM model to the 2D maximum intensity projection (MIP) of DBT images. In the proposed approach, the differences between the FFDM and DBT domains are reduced via HM and then the base model, which was trained on abundant FFDM images, is fine-tuned. When evaluating on image patches extracted around identified findings, we are able to achieve similar areas under the receiver operating characteristic curve (ROC AUC) of ∼ 0.9 for FFDM and ∼ 0.85 for MIP images, as compared to a ROC AUC of ∼ 0.75 when tested directly on MIP images.

show abstract

Section: Resultsmentioning

confidence: 99%

“…This inputdependent fine-tuning approach enables targeting layers per input instance and leads to better accuracy. 16 We refer readers to the original paper 16 for further details of SpotTune.…”

Section: Fine-tuningmentioning

confidence: 99%

Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis

Singh¹,

Matthews²,

Shah³

et al. 2020

Medical Imaging 2020: Computer-Aided Diagnosis

View full text Add to dashboard Cite

show abstract

“…In particular, while in [8] simple multiplicative binary masks are used to indicate which network parameters are useful for a new task and which are not, in [9] a more general formulation is proposed considering affine transformations. Guo et al [12] proposed an adaptive fine-tuning method and derive specialized classifiers by fine-tuning certain layers according to a given target image.…”

Section: Related Workmentioning

confidence: 99%

“…Interestingly, the best performing approach in terms of score, i.e. SpotTune [12], requires a much larger number of parameters that would restrict the use of this method when increasing the number of domains.…”

Section: Multi-domain Learningmentioning

confidence: 99%

Budget-Aware Adapters for Multi-Domain Learning

Berriel

Lathuilière

Nabi³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Multi-Domain Learning (MDL) refers to the problem of learning a set of models derived from a common deep architecture, each one specialized to perform a task in a certain domain (e.g. photos, sketches, paintings). This paper tackles MDL with a particular interest in obtaining domain-specific models with an adjustable budget in terms of the number of network parameters and computational complexity. Our intuition is that, as in real applications the number of domains and tasks can be very large, an effective MDL approach should not only focus on accuracy but also on having as few parameters as possible. To implement this idea we derive specialized deep models for each domain by adapting a pre-trained architecture but, differently from other methods, we propose a novel strategy to automatically adjust the computational complexity of the network. To this aim, we introduce Budget-Aware Adapters that select the most relevant feature channels to better handle data from a novel domain. Some constraints on the number of active switches are imposed in order to obtain a network respecting the desired complexity budget. Experimentally, we show that our approach leads to recognition accuracy competitive with state-of-the-art approaches but with much lighter networks both in terms of storage and computation. * Work done when with MHUG (DISI, University of Trento). 2

show abstract

“…Long et al [24] propose a deep adaptation network architecture to match the mean embeddings of different domain distributions in a reproducing kernel Hilbert space. Guo et al [13] propose an adaptive fine-tuning approach to find the optimal fine-tuning strategy per instance for the target data. Readers can refer to [27] and the references therein for details about transfer learning.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Deep Feature Transfer for Low Resolution Image Classification

Zhang

Wang

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

In this paper, we propose a simple while effective unsupervised deep feature transfer algorithm for low resolution image classification. No fine-tuning on convenet filters is required in our method. We use pre-trained convenet to extract features for both high-and low-resolution images, and then feed them into a two-layer feature transfer network for knowledge transfer. A SVM classifier is learned directly using these transferred low resolution features. Our network can be embedded into the state-of-the-art deep neural networks as a plug-in feature enhancement module. It preserves data structures in feature space for high resolution images, and transfers the distinguishing features from a wellstructured source domain (high resolution features space) to a not well-organized target domain (low resolution features space). Extensive experiments on VOC2007 test set show that the proposed method achieves significant improvements over the baseline of using feature extraction.

show abstract

SpotTune: Transfer Learning Through Adaptive Fine-Tuning

Cited by 362 publications

References 42 publications

Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis

Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis

Budget-Aware Adapters for Multi-Domain Learning

Unsupervised Deep Feature Transfer for Low Resolution Image Classification

Contact Info

Product

Resources

About