Fine Tuning U-Net for Ultrasound Image Segmentation: Which Layers?

Abstract

Fine-tuning a network which has been trained on a large dataset is an alternative to full training in order to overcome the problem of scarce and expensive data in medical applications. While the shallow layers of the network are usually kept unchanged, deeper layers are modified according to the new dataset. This approach may not work for ultrasound images due to their drastically different appearance. In this study, we investigated the effect of fine-tuning different layers of a U-Net which was trained on segmentation of natural images in breast ultrasound image segmentation. Tuning the contracting part and fixing the expanding part resulted in substantially better results compared to fixing the contracting part and tuning the expanding part. Furthermore, we showed that starting to finetune the U-Net from the shallow layers and gradually including more layers will lead to a better performance compared to fine-tuning the network from the deep layers moving back to shallow layers. We did not observe the same results on segmentation of X-ray images, which have different salient features compared to ultrasound, it may therefore be more appropriate to fine-tune the shallow layers rather than deep layers. Shallow layers learn lower level features (including speckle pattern, and probably the noise and artifact properties) which are critical in automatic segmentation in this modality.

Fine-Tuning Approach for Segmentation of Gliomas in Brain Magnetic Resonance Images with a Machine Learning Method to Normalize Image Differences among Facilities

“…Hence, it is reasonable to make layer groups that consist of a shallow layer block and a deep layer block, which is directly connected to the shallow layer. Our results are compatible with the results obtained by Amiri et al [39].…”

“…Although many researchers recognize that domain shifts reduce the performance of machine learning models, this problem has been poorly investigated [ 38 , 39 , 40 ]. One of the main reasons for poor investigation is the requirement of datasets from multiple facilities.…”

“…Although the network architecture of U-Net is frequently used when building machine learning models for segmentation, it has rarely been investigated for fine-tuning. Amiri et al recently assessed the effects of fine-tuning different layers of the U-Net model that was trained on 1000 segmented natural images for segmentation of breast ultrasound images [ 39 ], and concluded that setting the shallow layer as a learnable layer was more effective than setting the deep layer. They also mentioned that concordant results were not observed in the segmentation of radiographs.…”

U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications

“…Nerve segmentation [151], [222] Base U-net [50] Inception block [223] Residual block [224] Modified parallel U-net Breast lesion [225] Base U-net [39] Attention gate Arterial wall [226] Base U-net Cardiovascular structures [227] Base U-net Fetal head [219] Base U-net Gastrointestinal tumor [228] Base U-net Knee cartilage [96] Modified U-net with dual parallel encoders Preterm birth prediction [220] Base U-net Thyroid [229] Residual block Transcranial detection [221] Base U-net Ovary detection [230] Base U-net…”

Dense Pixel-Labeling For Reverse-Transfer And Diagnostic Learning On Lung Ultrasound For Covid-19 And Pneumonia Detection

“…So the use of a semantic-segmentation model for diagnostic classification meets our criteria for reverse transfer learning. In the case of ultrasound, we feel our approach may be particularly useful as traditional transfer learning has proven challenging on ultrasound [14], Particular ultrasound challenges include speckle noise, confounding causes of particular pixel values, view-point dependence, and overall ambiguity in image interpretation. As a result, gradient descent over images from a few hundred, patients may get stuck by initially learning poor, over-fit features that may be correlated without being causal (e.g., learning that chest-wall fat-to-muscle ratio is a good body-mass-index disease predictor).…”