Spherical U-Net on Cortical Surfaces: Methods and Applications

Abstract

Convolutional Neural Networks (CNNs) have been providing the state-of-the-art performance for learning-related problems involving 2D/3D images in Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have a spherical topology in a manifold space, e.g., brain cortical or subcortical surfaces represented by triangular meshes, with large inter-subject and intrasubject variations in vertex number and local connectivity. Hence, there is no consistent neighborhood definition and thus no straightforward convolution/transposed convolution operations for cortical/subcortical surface data. In this paper, by leveraging the regular and consistent geometric structure of the resampled cortical surface mapped onto the spherical space, we propose a novel convolution filter analogous to the standard convolution on the image grid. Accordingly, we develop corresponding operations for convolution, pooling, and transposed convolution for spherical surface data and thus construct spherical CNNs. Specifically, we propose the Spherical U-Net architecture by replacing all operations in the standard U-Net with their spherical operation counterparts. We then apply the Spherical U-Net to two challenging and neuroscientifically important tasks in infant brains: cortical surface parcellation and cortical attribute map development prediction. Both applications demonstrate the competitive performance in the accuracy, computational efficiency, and effectiveness of our proposed Spherical U-Net, in comparison with the state-of-the-art methods.

Deep learning identifies morphological determinants of sex differences in the pre-adolescent brain

“…Now identifying cues differentiating between groups highly depends on the ‘stiffness’ of the deformation field ( Murphy et al, 2016 ; Wittek et al, 2010 ), which can substantially modify the shape and appearance of brain structures. One possible data driven approach for setting the stiffness with respect to the cortex is to first parcellate the structure (via a surface based segmentation tool, Dale et al, 1999 ; Onofrey et al, 2018 ; Zhao et al, 2019 ) and then perform an ROI-based registration for the whole brain (such as Lopez-Garcia et al, 2006 ; Yi et al, 2006 ). As any of these registration can negatively affect analysis, their effect on our deep learning findings needs to be further investigated.…”

Intrinsic Patch-Based Cortical Anatomical Parcellation Using Graph Convolutional Neural Network on Surface Manifold

“…Of note, the advantage of our method is that it does not need any spherical mapping and registration. Although the methods in Wu et al [6] and Zhao et al [7] also do not rely on spherical registration, they still need to map cortical surfaces onto a sphere. They are not applicable on impaired cortical surfaces with non-spherical topology, because spherical mapping is sensitive to topological noises and cortical surfaces need to be topologically correct before mapping.…”

“…E.g., the functional network studies need the parcellation ROIs to investigate the connections across different brain regions. Previously, many computational methods have been proposed [1,[3][4][5][6][7][8][9] for automatic cortical surface parcellation. Typically, these methods include three major steps.…”

Harmonization of Infant Cortical Thickness Using Surface-to-Surface Cycle-Consistent Adversarial Networks

“…We use the spherical U-Net [11] architecture as our generative network. Leveraging the spherical topology of cortical surfaces, the spherical U-Net first extends convolution, pooling, and upsampling operations to the spherical space using DiNe filter on regularly resampled spherical surfaces, and then constructs U-Net using corresponding spherical operations.…”

“…While not developed explicitly for harmonization, a number of recently developed deep learning techniques [11,12] could potentially be adapted to address these issues. First, the spherical U-Net architecture [11] provides an effective Direct Neighbor (DiNe) filter to extend conventional convolutional neural network (CNN) to the cortical surface with an inherent spherical topology. It was originally designed for cortical surface parcellation [10] and achieves state-of-the-art performance, which could be used as a generator for site-to-site cortical surface property map translation.…”