Harmonization of Infant Cortical Thickness Using Surface-to-Surface Cycle-Consistent Adversarial Networks

Abstract

Increasing multi-site infant neuroimaging datasets are facilitating the research on understanding early brain development with larger sample size and bigger statistical power. However, a joint analysis of cortical properties (e.g., cortical thickness) is unavoidably facing the problem of nonbiological variance introduced by differences in MRI scanners. To address this issue, in this paper, we propose cycle-consistent adversarial networks based on spherical cortical surface to harmonize cortical thickness maps between different scanners. We combine the spherical U-Net and CycleGAN to construct a surface-to-surface CycleGAN (S2SGAN). Specifically, we model the harmonization from scanner X to scanner Y as a surface-to-surface translation task. The first goal of harmonization is to learn a mapping G X : X → Y such that the distribution of surface thickness maps from G X (X) is indistinguishable from Y. Since this mapping is highly under-constrained, with the second goal of harmonization to preserve individual differences, we utilize the inverse mapping G Y : Y → X and the cycle consistency loss to enforce G Y (G X (X)) ≈ X (and vice versa). Furthermore, we incorporate the correlation coefficient loss to guarantee the structure consistency between the original and the generated surface thickness maps. Quantitative evaluation on both synthesized and real infant cortical data demonstrates the superior ability of our method in removing unwanted scanner effects and preserving individual differences simultaneously, compared to the state-of-the-art methods.

AVN: An Adversarial Variation Network Model for Handwritten Signature Verification

“…It utilizes a cycle consistency loss to drive the mappings between the original image and the target image to be cycle consistent. This mechanism was later applied to numerous other tasks [49], [50], [51], such as image-text matching [52], domain adaptation [53], and video understanding [44]. Different from the previous models, our method introduces the new variation consistency mechanism to train a stronger discriminative model.…”

Style Transfer Using Generative Adversarial Networks for Multi-Site MRI Harmonization

“…To quantify inter-subject differences, we used the intensity differences to compute Euclidean distances (Zhao et al, 2019) , where n = 10 is the number of scans, and I the whole-image voxel intensity vectors for scans i and j. The goal was to estimate how the distances were preserved relative to each other before and after harmonization.…”

Deep Learning-Based Unlearning of Dataset Bias for MRI Harmonisation and Confound Removal

“…In addition to the variational autoencoders used in [36], other generative models have been used to harmonise MRI, largely based on deep learning models including U-Net [39] based models and cycleGAN [10,56] based models [55]. These are limited by needing either paired or 'travelling heads' data for training for each site, which is expensive and infeasible to acquire in large numbers, but hard to evaluate without them [36].…”

“…Placed between the feature extractor and domain classifier, the gradient reversal layer acts as an identity function in the forward step, and during the backward pass, it multiplies the gradient function by −λ, where λ is a hyperparameter set empirically for a given experimental setup [18]. The extension of domain adaptation to N source domains -key to enable the harmonisation of more than two scanners -was formalised in [4] and demonstrated for adversarial domain adaptation in [55].…”