Diffeomorphic registration with intensity transformation and missing data: Application to 3D digital pathology of Alzheimer's disease

Abstract

This paper examines the problem of diffeomorphic image mapping in the presence of differing image intensity profiles and missing data. Our motivation comes from the problem of aligning 3D brain MRI with 100 micron isotropic resolution, to histology sections with 1 micron in plane resolution. Multiple stains, as well as damaged, folded, or missing tissue are common in this situation. We overcome these challenges by introducing two new concepts. Cross modality image matching is achieved by jointly estimating polynomial transformations of the atlas intensity, together with pose and deformation parameters. Missing data is accommodated via a multiple atlas selection procedure where several atlases may be of homogeneous intensity and correspond to "background" or "artifact". The two concepts are combined within an Expectation Maximization algorithm, where atlas selection posteriors and deformation parameters are updated iteratively, and polynomial coefficients are computed in closed form. We show results for 3D reconstruction of digital pathology and MRI in standard atlas coordinates. In conjunction with convolutional neural networks, we quantify the 3D density distribution of tauopathy throughout the medial temporal lobe of an Alzheimer's disease postmortem specimen. Author summaryOur work in Alzheimer's disease (AD) is attempting to connect histopathology at autopsy and longitudinal clinical magnetic resonance imaging (MRI), combining the strengths of each modality in a common coordinate system. We are bridging this gap by January 4, 2019 1/19 using post mortem high resolution MRI to reconstruct digital pathology in 3D. This image registration problem is challenging because it combines images from different modalities in the presence of missing tissue and artifacts. We overcome this challenge by developing a new registration technique that simultaneously classifies each pixel as "good data" / "missing tissue" / "artifact", learns a contrast transformation between modalities, and computes deformation parameters. We name this technique "(D)eformable (R)egistration and (I)ntensity (T)ransformation with (M)issing (D)ata", pronounced as "Dr. It, M.D.". In conjunction with convolutional neural networks, we use this technique to map the three dimensional distribution of tau tangles in the medial temporal lobe of an AD postmortem specimen. 1 High throughput neuroinformatics is emerging in neuroscience [1, 2]. Atlas based image 2 analysis plays a key role, as it enables information encoded by millions of independent 3 voxel measurements to be reconstructed in ontologies of the roughly 100 evolutionarily 4 stable structures. At the 1 millimeter scale there are many atlases, including Tailarach 5 coordinates [3], Montreal Neurological Institute (MNI) [4], and Mori's diffusion tensor 6 imaging (DTI) white matter atlases [5], which define the locations of neuroanatomical 7 structures as well as important structural and functional properties such as volume, 8 shape, blood oxygen-level dependent (BOLD) signals, etc. At micron and m...

Visualizing synaptic plasticity in vivo by large-scale imaging of endogenous AMPA receptors

“…Recently, deep-learning-based systems have achieved state-of-the-art performance in analyzing microscopy images (Moen et al, 2019). Many different architectures have been used, including our work detecting tau tangles with sliding windows to annotate single pixels (Tward et al, 2020), UNET (Ronneberger et al, 2015) for annotating larger blocks which has been implemented in FIJI (Falk et al, 2019), and other elaborations such as VNET (Milletari et al, 2016). Typically, trained networks assign class probabilities to each pixel, which are collected into larger objects based on connected components or watershed approaches available in standard packages, such as FIJI (Schindelin et al, 2012).…”

Virtual mouse brain histology from multi-contrast MRI via deep learning

“…The present study focused on inferring maps of key cellular structures in the mouse brain from multi-contrast MRI data. Previous works on this problem include: new MRI contrasts that capture specific aspects of cellular structures of interest (19,20); carefully constructed tissue models for MR signals (21); statistical methods to extract relevant information from multi-contrast MRI (2); and techniques to register histology and MRI data (22,23) for validation (24,25). Here, we built on these efforts by demonstrating that deep learning networks trained by co-registered histological and MRI data can improve our ability to detect target cellular structures.…”

Projective LDDMM: Spatially Reconstructing a Story of Rostrally-Dominant Tau in Alzheimer’s Disease

“…As described previously [62, 36], we used a convolutional neural network to model and predict probabilities of being part of a tau tangle for each pixel in a digital histology image. To capture larger contextual features as well as local information for producing per pixel probabilities at high resolutions, we trained UNETs [63] with the architecture described in Table F2 (see Appendix F).…”