Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts

Valvano, Gabriele; Leo, Andrea; Tsaftaris, Sotirios A.

doi:10.48550/arxiv.2108.11926

“…In fact, average reconstruction error is approximately 0.007 on the training set and 0.011 on the test set. Inspired by [13,37] we perform testtime training (TTT) to better reconstruct by fine-tuning the reconstruction loss L rec (X, X) to update F ψ and R ω with the kernels and T θ fixed. This should in turn produce better vMF likelihoods.…”

Section: Test-time Domain Generalisationmentioning

confidence: 99%

vMFNet: Compositionality Meets Domain-Generalised Segmentation

Liu¹,

Thermos²,

Sánchez³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Training medical image segmentation models usually requires a large amount of labeled data. By contrast, humans can quickly learn to accurately recognise anatomy of interest from medical (e.g. MRI and CT) images with some limited guidance. Such recognition ability can easily generalise to new images from different clinical centres. This rapid and generalisable learning ability is mostly due to the compositional structure of image patterns in the human brain, which is less incorporated in medical image segmentation. In this paper, we model the compositional components (i.e. patterns) of human anatomy as learnable von-Mises-Fisher (vMF) kernels, which are robust to images collected from different domains (e.g. clinical centres). The image features can be decomposed to (or composed by) the components with the composing operations, i.e. the vMF likelihoods. The vMF likelihoods tell how likely each anatomical part is at each position of the image. Hence, the segmentation mask can be predicted based on the vMF likelihoods. Moreover, with a reconstruction module, unlabeled data can also be used to learn the vMF kernels and likelihoods by recombining them to reconstruct the input image. Extensive experiments show that the proposed vMFNet achieves improved generalisation performance on two benchmarks, especially when annotations are limited. Code is publicly available at: https://github.com/vios-s/vMFNet.

show abstract

“…They first simulate shifted domain images via a randomly weighted shallow network; then they intervene upon the images such that spurious correlations are removed and finally train their segmentation model while enforcing a domain invariance condition. [Valvano et al, 2021] develop a method to reuse adversarial mask discriminators for test-time training to combat distribution shifts in medical image segmentation tasks. In their discussion of their method they explain the good performance of their method under a causal lens.…”

Section: Domain Generalizationmentioning

confidence: 99%

A Review of Causality for Learning Algorithms in Medical Image Analysis

Vlontzos¹,

Rueckert²,

Kainz³

2022

Preprint

View full text Add to dashboard Cite

Medical image analysis is a vibrant research area that offers doctors and medical practitioners invaluable insight and the ability to accurately diagnose and monitor disease. Machine learning provides an additional boost for this area. However, machine learning for medical image analysis is particularly vulnerable to natural biases like domain shifts that affect algorithmic performance and robustness. In this paper we analyze machine learning for medical image analysis within the framework of Technology Readiness Levels and review how causal analysis methods can fill a gap when creating robust and adaptable medical image analysis algorithms. We review methods using causality in medical imaging AI/ML and find that causal analysis has the potential to mitigate critical problems for clinical translation but that uptake and clinical downstream research has been limited so far.

show abstract

vMFNet: Compositionality Meets Domain-generalised Segmentation

Liu¹,

Thermos²,

Sánchez³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Training medical image segmentation models usually requires a large amount of labeled data. By contrast, humans can quickly learn to accurately recognise anatomy of interest from medical (e.g. MRI and CT) images with some limited guidance. Such recognition ability can easily generalise to new images from different clinical centres. This rapid and generalisable learning ability is mostly due to the compositional structure of image patterns in the human brain, which is less incorporated in medical image segmentation. In this paper, we model the compositional components (i.e. patterns) of human anatomy as learnable von-Mises-Fisher (vMF) kernels, which are robust to images collected from different domains (e.g. clinical centres). The image features can be decomposed to (or composed by) the components with the composing operations, i.e. the vMF likelihoods. The vMF likelihoods tell how likely each anatomical part is at each position of the image. Hence, the segmentation mask can be predicted based on the vMF likelihoods. Moreover, with a reconstruction module, unlabeled data can also be used to learn the vMF kernels and likelihoods by recombining them to reconstruct the input image. Extensive experiments show that the proposed vMFNet achieves improved generalisation performance on two benchmarks, especially when annotations are limited. Code is publicly available at: https://github.com/vios-s/vMFNet.

show abstract

Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts

Cited by 3 publications

References 41 publications

vMFNet: Compositionality Meets Domain-Generalised Segmentation

vMFNet: Compositionality Meets Domain-Generalised Segmentation

A Review of Causality for Learning Algorithms in Medical Image Analysis

vMFNet: Compositionality Meets Domain-generalised Segmentation

Contact Info

Product

Resources

About