Multi-Scale Modeling of Neural Structure in X-Ray Imagery

Balwani, Aishwarya H.; Miano, Joseph D.; Liu, Ran; Kitchell, Lindsey; Prasad, Judy A.; Johnson, Erik C.; Gray-Roncal, William; Dyer, Eva L.

doi:10.1109/icip42928.2021.9506174

Cited by 4 publications

(2 citation statements)

References 24 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These datasets have the micro-or nanoscale resolution and large spatial extents required to resolve subcellular structures (e.g., mitochondria and synapses), microstructures (e.g., glia, neurons, and vasculature), and macrostructure (e.g., brain regions, cortical layer structure, and long-range white matter projections). The multi-scale nature and large size of these datasets requires new ML tools (16), which drives the need for benchmarks that can extract representations of neural structure at different scales.…”

Section: The Need For a Benchmark In Brain Mapping And Connectomicsmentioning

confidence: 99%

“…When analyzing imaging data that spans many different brain regions, one important question is the degree to which global image features correlate with the brain region from which the sample is drawn (16). Thus, we can pose this as a classification problem, where we pull a small patch (or small region) from the data and estimate which of the 4 brain regions the sample was drawn from.…”

Section: Task 1: Image-level Classification Of Brain Region-of-intere...mentioning

confidence: 99%

See 1 more Smart Citation

MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstraction

Quesada¹,

Sathidevi²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and selfsupervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/.

show abstract

Section: The Need For a Benchmark In Brain Mapping And Connectomicsmentioning

confidence: 99%

Section: Task 1: Image-level Classification Of Brain Region-of-intere...mentioning

confidence: 99%

MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstraction

Quesada¹,

Sathidevi²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

A three-dimensional thalamocortical dataset for characterizing brain heterogeneity

Prasad

Balwani

Johnson

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Neural cytoarchitecture is heterogeneous, varying both across and within brain regions. The consistent identification of regions of interest is one of the most critical aspects in examining neurocircuitry, as these structures serve as the vital landmarks with which to map brain pathways. Access to continuous, three-dimensional volumes that span multiple brain areas not only provides richer context for identifying such landmarks, but also enables a deeper probing of the microstructures within. Here, we describe a three-dimensional X-ray microtomography imaging dataset of a well-known and validated thalamocortical sample, encompassing a range of cortical and subcortical structures. In doing so, we provide the field with access to a micron-scale anatomical imaging dataset ideal for studying heterogeneity of neural structure. Background and SummaryWhether focusing on a large swath of cortex or a single subcortical nucleus, consistent and reliable visualization of cytoarchitecture is critical for the creation of reference points which demarcate the brain's landscape [1]. This is true not only for the identification of landmarks (or regions of interest), but also the study of local circuits therein. Indeed, it is the distinguishing features in brain cytoarchitecture which arise at small, local scales (i.e., through clusters of cells which are packed in discrete barrels or layers; see [2,3]) that continue to emerge across larger spatial scales to reveal the presence of functionally distinct regions. Thus, detailed views into the brain's architecture can be used to experimentally manipulate circuits, and to advance the field's understanding and integration of each of these overarching systems.With advances in the reconstruction and analysis of significantly larger brain volumes, neuroscientists are now able to visualize patterns of microarchitecture that arise at a scale previously inaccessible using traditional methods [4,5,6]. Examples such as CLARITY [7], expansion microscopy [8], serial two photon tomography [9, 10, 11], multi-beam scanning electron microscopy [12], and X-ray microtomography [13,14,15,16,17], now provide access to several regions of interest within a volume of tissue simultaneously, providing rich context to study both local circuitry and long-range projections. With many of these new techniques, it is possible to image and analyze large intact anatomical samples that preserve the connectivity between multiple regions of interest [18,19], thus providing a lens into the heterogeneity of neural structure within and across different brain areas.Here, we introduce a three-dimensional neuroanatomical dataset extracted from a validated, in-vitro mouse thalamocortical sample spanning six anatomically distinct regions of interest (somatosensory cortex, two thalamic nuclei, zona incerta, striatum and hypothalamus) [18]. This dataset was reconstructed using X-ray microtomography to reveal a diverse composition of microstructures (e.g., myelinated axons, cell bodies, and vasculature) within each region at isotr...

show abstract

A Deep Feature Learning Approach for Mapping the Brain’s Microarchitecture and Organization

Balwani

Dyer

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Models of neural architecture and organization are critical for the study of disease, aging, and development. Unfortunately, automating the process of building maps of microarchitectural differences both within and across brains still remains a challenge. In this paper, we present a way to build data-driven representations of brain structure using deep learning. With this model we can build meaningful representations of brain structure within an area, learn how different areas are related to one another anatomically, and use this model to discover new regions of interest within a sample that share similar characteristics in terms of their anatomical composition. We start by training a deep convolutional neural network to predict the brain area that it is in, using only small snapshots of its immediate surroundings. By requiring that the network learn to discriminate brain areas from these local views, it learns a rich representation of the underlying anatomical features that allow it to distinguish different brain areas. Once we have the trained network, we open up the black box, extract features from its last hidden layer, and then factorize them. After forming a low-dimensional factorization of the network's representations, we find that the learned factors and their embeddings can be used to further resolve biologically meaningful subdivisions within brain regions (e.g., laminar divisions and barrels in somatosensory cortex). These findings speak to the potential use of neural networks to learn meaningful features for modeling neural architecture, and discovering new patterns in brain anatomy directly from images.Much of our study of the brain and it's organization, which dates back to Cajal's beautiful renderings of neural architecture [10], still relies on human reasoning to define regions-ofinterest (ROIs). This is done by examining different parts of an image, either in terms of their anatomical compositions or patterning, and then building an atlas or model of how the architecture changes across different brain regions. Moving forward however, given the everincreasing sizes of new neuroimaging datasets, we need automated solutions that can find characteristics of brain structure, discover architectural patterns or primitives that are characteristic of a brain area, and also provide good ways to discover substructures (like cortical lamina) within a known brain area.Unfortunately, when attempting to translate expert human knowledge into an automated method for region discovery, it is often unclear how to define the necessary image properties to extract. Moreover, extracting these features can be difficult to automate, or they may not be robust to minor changes in data (e.g., illumination conditions, background intensity, or noise) [11,12]. Features therefore typically need to be hand-crafted for each individual dataset and application, and problems with generalization are only further exacerbated by biological variability, as well as process variations introduced during sample preparation, imaging, or post...

show abstract

Multi-Scale Modeling of Neural Structure in X-Ray Imagery

Cited by 4 publications

References 24 publications

MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstraction

MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstraction

A three-dimensional thalamocortical dataset for characterizing brain heterogeneity

A Deep Feature Learning Approach for Mapping the Brain’s Microarchitecture and Organization

Contact Info

Product

Resources

About