Morphology-Preserving Autoregressive 3D Generative Modelling of the Brain

Abstract

Human anatomy, morphology, and associated diseases can be studied using medical imaging data. However, access to medical imaging data is restricted by governance and privacy concerns, data ownership, and the cost of acquisition, thus limiting our ability to understand the human body. A possible solution to this issue is the creation of a model able to learn and then generate synthetic images of the human body conditioned on specific characteristics of relevance (e.g., age, sex, and disease status). Deep generative models, in the form of neural networks, have been recently used to create synthetic 2D images of natural scenes. Still, the ability to produce high-resolution 3D volumetric imaging data with correct anatomical morphology has been hampered by data scarcity and algorithmic and computational limitations. This work proposes a generative model that can be scaled to produce anatomically correct, high-resolution, and realistic images of the human brain, with the necessary quality to allow further downstream analyses. The ability to generate a potentially unlimited amount of data not only enables large-scale studies of human anatomy and pathology without jeopardizing patient privacy, but also significantly advances research in the field of anomaly detection, modality synthesis, learning under limited data, and fair and ethical AI. Code and trained models are available at: https://github.com/AmigoLab/SynthAnatomy.

Latent Transformer Models for out-of-distribution detection

“…The trained transformer can be used to quantify the likelihood of a new sample, with samples being rejected as OOD using a one-sided threshold on this likelihood. This architecture has been used to enable high-resolution images synthesis in 2D ( Esser et al, 2021 ) and 3D ( Tudosiu et al, 2020 , Tudosiu et al, 2022 ), and to perform unsupervised pathology detection ( Pinaya et al, 2021 , Pinaya et al, 2022 ), but, to our knowledge, this work is the first time they have been demonstrated for whole-image OOD.…”

“…In this work, we term these Latent Transformer Models (LTM) in analogy to Latent Diffusion Models (LDM) that use a similar latent backbone to train diffusion models ( Rombach et al, 2022 , Sohl-Dickstein et al, 2015 , Ho et al, 2020 ). LTMs have achieved state-of-the-art unsupervised pathology segmentation for 2D and 3D medical images ( Pinaya et al, 2021 , Pinaya et al, 2022 ) and can produce high-quality 3D generative images of the brain ( Tudosiu et al, 2020 , Tudosiu et al, 2022 ). These results suggest that LTMs might be applied to fully 3D OOD detection but, to our knowledge, no published work is attempting this.…”

Opportunities and obstacles in non-invasive brain stimulation

“…Achieving such precision involves structural imaging, simulation, and neuronavigation, a resource-intensive process. More scalable approaches are needed, potentially combining subject-specific MRI approximated with generative modeling (Tudosiu et al, 2022 ) and utilizing smartphone-based 3D head scanning (Everitt et al, 2023 ). Additionally, techniques such as electrical impedance tomography and pulse-echo ultrasound can estimate the electrical and acoustic properties of the head, respectively (Fernández-Corazza et al, 2017 ; He et al, 2021 ).…”