Deep transfer learning for cerebral cortex using area-preserving geometry mapping

Abstract

Limited sample size hinders the application of deep learning in brain image analysis, and transfer learning is a possible solution. However, most pretrained models are 2D based and cannot be applied directly to 3D brain images. In this study, we propose a novel framework to apply 2D pretrained models to 3D brain images by projecting surface-based cortical morphometry into planar images using computational geometry mapping. Firstly, 3D cortical meshes are reconstructed from magnetic resonance imaging (MRI) using FreeSurfer and projected into 2D planar meshes with topological preservation based on area-preserving geometry mapping. Then, 2D deep models pretrained on ImageNet are adopted and fine-tuned for cortical image classification on morphometric shape metrics. We apply the framework to sex classification on the Human Connectome Project dataset and autism spectrum disorder (ASD) classification on the Autism Brain Imaging Data Exchange dataset. Moreover, a 2-stage transfer learning strategy is suggested to boost the ASD classification performance by using the sex classification as an intermediate task. Our framework brings significant improvement in sex classification and ASD classification with transfer learning. In summary, the proposed framework builds a bridge between 3D cortical data and 2D models, making 2D pretrained models available for brain image analysis in cognitive and psychiatric neuroscience.

Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis

“…all contributed to the potential high ROB of these AI models in the analysis domain ( Table 1 ). Although almost all of the models were rated with high ROB on calibration and data complexities issues, 52 models 27 , 28 , 31 , 42 , 51 , 61 , 70 , 82 , 84 , 86 , 93 , 112 , 132 , 139 , 171 , 184 , 213 , 228 , 231 , 243 , 244 , 248 , 250 , 254 , 255 , 261 , 263 , 266 , 269 , 271 , 272 , 282 , 323 , 337 , 346 , 371 , 422 , 436 , 437 , 438 , 463 , 474 , 476 , 481 , 486 , 506 , 514 , 518 , 534 , 535 were still rated with low overall ROB in the analysis domain because these issues were judged as having limited implications for the models.…”

Machine learning for autism spectrum disorder diagnosis using structural magnetic resonance imaging: Promising but challenging

“…The prefrontal cortex and cerebellum ( Gao et al, 2022 ; Wang et al, 2021 ) and the temporal cortex ( Wang et al, 2021 ) help classify structural covariance brain networks. Basic conscious motions are physically and functionally linked to the prefrontal cortex, and its abnormality is associated with ASD’s emotional and social domain.…”

“…In addition, the cerebellum is essential for cognitive functions, memory, emotion, and language ( Gao et al, 2022 ). The ASD and HC groups differed significantly in the choroid plexus, cuneus, left putamen, and cerebellar cortex ( Pinaya et al, 2019 ).…”

“…However, they used a small data set and did not depict potential biomarkers. The study ( Gao et al, 2022 ) addressed this issue by identifying a biomarker using a Res-Net and gradient-weighted class activation mapping (Grad-CAM) on individual structural covariance networks. However, like most approaches to ASD diagnosis that have focused on features recovered from a separate ROIs, non-local relationships that are contradict brain network evidence have been ignored.…”

“…In Wang et al (2021) , a transformer-based DL architecture for more stable self-attention is presented; this self-attention DL model employs individual MBNs instead of raw MRI to identify ASD. Grad-CAM heat maps are hierarchical, like CNN’s simple-to-complex feature extraction algorithm ( Gao et al, 2022 ). However, in Wang et al (2021) , the maps of self-attention coefficients in the first and second layers are similar, indicating a consistent diagnosis.…”