A Deep Feature Learning Approach for Mapping the Brain’s Microarchitecture and Organization

Abstract

Models of neural architecture and organization are critical for the study of disease, aging, and development. Unfortunately, automating the process of building maps of microarchitectural differences both within and across brains still remains a challenge. In this paper, we present a way to build data-driven representations of brain structure using deep learning. With this model we can build meaningful representations of brain structure within an area, learn how different areas are related to one another anatomically, and use this model to discover new regions of interest within a sample that share similar characteristics in terms of their anatomical composition. We start by training a deep convolutional neural network to predict the brain area that it is in, using only small snapshots of its immediate surroundings. By requiring that the network learn to discriminate brain areas from these local views, it learns a rich representation of the underlying anatomical features that allow it to distinguish different brain areas. Once we have the trained network, we open up the black box, extract features from its last hidden layer, and then factorize them. After forming a low-dimensional factorization of the network's representations, we find that the learned factors and their embeddings can be used to further resolve biologically meaningful subdivisions within brain regions (e.g., laminar divisions and barrels in somatosensory cortex). These findings speak to the potential use of neural networks to learn meaningful features for modeling neural architecture, and discovering new patterns in brain anatomy directly from images.Much of our study of the brain and it's organization, which dates back to Cajal's beautiful renderings of neural architecture [10], still relies on human reasoning to define regions-ofinterest (ROIs). This is done by examining different parts of an image, either in terms of their anatomical compositions or patterning, and then building an atlas or model of how the architecture changes across different brain regions. Moving forward however, given the everincreasing sizes of new neuroimaging datasets, we need automated solutions that can find characteristics of brain structure, discover architectural patterns or primitives that are characteristic of a brain area, and also provide good ways to discover substructures (like cortical lamina) within a known brain area.Unfortunately, when attempting to translate expert human knowledge into an automated method for region discovery, it is often unclear how to define the necessary image properties to extract. Moreover, extracting these features can be difficult to automate, or they may not be robust to minor changes in data (e.g., illumination conditions, background intensity, or noise) [11,12]. Features therefore typically need to be hand-crafted for each individual dataset and application, and problems with generalization are only further exacerbated by biological variability, as well as process variations introduced during sample preparation, imaging, or post...

Interpretable machine learning as a tool for scientific discovery in chemistry

“…The exploration of the potential of interpretable ML to the sciences is growing, with applications in genomics, 46 manybody systems, 47 neuroscience 48 and chemistry. Although this chemical review has focused on interpretation with respect to drug discovery and quantum chemistry, the potential of ML has been explored in other areas of chemistry, such as the use of ML for computational heterogeneous catalysis 49 and retrosynthesis, 50 and the use of interpretable ML in these and other fields is expected to prove to be immensely useful.…”

Multi-Scale Modeling of Neural Structure in X-Ray Imagery

“…Even though our proposed architecture has high macroscale prediction accuracy, we find that there often are small clusters throughout the output volume that are misclassified. We therefore enforce local spatial consensus in 3D amongst macroscale predictions by extending a 2D k-Nearest Neighbours (kNN) based cleanup procedure [15] (Methods, Algorithm 1) and apply it to the predicted brain-area segmentations of our model. The working of the algorithm is as follows: Assuming that the i th ROI has m i connected components, we only maintain the set of points in the m i largest components predicted with the label i, which we call our "seeds".…”

“…In this work we present a deep learning approach for modelling neural architecture across multiple spatial scales in a data-driven and generalizable manner, using a multi-task learning based framework. Specifically, we leverage softparameter sharing to combine a model of the macrostructure, that is learnt by discriminating between different brain areas in the sample [14,15], with that of the microstructure, which is learnt by semantically segment different neural components present in the sample, into a single parameterized framework trained end-to-end with a two-step, warm start procedure. We empirically show that our proposed multi-task architecture and training methodology consistently perform at the level of, or better than strong single-task baselines at both scales.…”