Autoencoder based self-supervised test-time adaptation for medical image analysis

He, Yufan; Carass, Aaron; Zuo, Lianrui; Prince, Jerry L.

doi:10.1016/j.media.2021.102136

Cited by 57 publications

(56 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these three settings, DG is the most appealing -contrary to other settings, it does not require sharing data between institutions or annotating additional images. Recently, two new settings have been proposed -source-free domain adaptation (SFDA) [10] and test-time adaptation (TTA) [11], [12]. Here, the performance of CNNs trained with DG techniques is further improved by adapting them using unlabelled image(s) from the test distribution.…”

Section: Categories Of Methods To Tackle the Ds Problemmentioning

confidence: 99%

“…Slab B shows the mapping of the inputs (green) to the outputs (purple), via the normalized features (blue). Slab A shows the training of prior models (autoencoder (center) [12], denoising autoencoder (right) [11]) to be used for TTA: the AE is trained to auto-encode features of training images and the DAE is trained to denoise corrupted outputs (from a specific corruption distribution indicated by the crescent). Finally, slab C shows the desirable behaviour (pink arrows) and potential failure cases (red arrows) when the trained prior models are used to guide TTA.…”

Section: The Distribution Shift Problemmentioning

confidence: 99%

“…segmentation, deformation field, enhanced image). The image-specific parameters, φ, are adapted by requiring adherence to a prior model, H ψ , either in the output space [11] or in the feature space [12]. H ψ encourages similarity between outputs or features of the test image with those of the training images.…”

Section: Test-time Adaptationmentioning

confidence: 99%

“…H ψ encourages similarity between outputs or features of the test image with those of the training images. It is itself modelled using a CNN and trained in a self-supervised manner -as a denoising autoencoder (DAE) in [11] and as an autoencoder (AE) in [12].…”

Section: Test-time Adaptationmentioning

confidence: 99%

“…Indeed, this is symptomatic of the CNN's poor performance on OOD images. The main assumption of TTA methods like [11], [12] is that H ψ is capable of mapping such features (outputs) to ones that are similar to features (outputs) observed during training. However, if H ψ is modelled with a CNN, it is likely to be vulnerable to a DS problem of its own -that is, the outputs of H ψ may be unreliable when its test inputs are from a different distribution as compared to its training inputs.…”

Section: The Ds Problem In H ψmentioning

confidence: 99%

See 4 more Smart Citations

A Field of Experts Prior for Adapting Neural Networks at Test Time

Karani¹,

Brunner²,

Erdil³

et al. 2022

Preprint

View full text Add to dashboard Cite

Supervised learning methods based on convolutional neural networks (CNNs) show promising performance in several medical image analysis tasks. Such performance, however, is marred in the presence of acquisition-related distribution shifts between training and test images. Recently, it has been proposed to tackle this problem by fine-tuning trained CNNs for each test image. Such test-time-adaptation (TTA) is a promising and practical strategy for improving robustness to distribution shifts as it requires neither data sharing between institutions nor annotating additional data. Previous TTA methods use a helper model to increase similarity between outputs and/or features extracted from a test image with those of the training images. Such helpers, which are typically modeled using CNNs and trained in a self-supervised manner, can be task-specific and themselves vulnerable to distribution shifts in their inputs. To overcome these problems, we propose to carry out TTA by matching the feature distributions of test and training images, as modelled by a field-of-experts (FoE) prior. FoEs model complicated probability distributions as products of several simpler expert distributions. We use the 1D marginal distributions of a trained task CNN's features as the experts in the FoE model. Further, we carry out principal component analysis (PCA) of patches of the task CNN's features, and consider the distributions of the PCA loadings as additional experts. We extensively validate the method's efficacy on 5 MRI segmentation tasks (healthy tissues in 4 anatomical regions and lesion segmentation in 1 one anatomy), using data from 17 institutions, and on a MRI registration task, using data from 3 institutions. We find that the proposed FoE-based TTA is generically applicable in multiple tasks, and outperforms all previous TTA methods for lesion segmentation. For healthy tissue segmentation, the proposed method outperforms other task-agnostic TTA methods, but a previous TTA method which is specifically designed for segmentation performs the best for most of the tested datasets. Our implementation is publicly available here.

show abstract

Section: Categories Of Methods To Tackle the Ds Problemmentioning

confidence: 99%

Section: The Distribution Shift Problemmentioning

confidence: 99%