Coarse Graining of Data via Inhomogeneous Diffusion Condensation

Brugnone, Nathan; Krishnaswamy, Smita; Gonopolskiy, Alex; Moyle, Mark W.; Kuchroo, Manik; Dijk, David van; Moon, Kevin R.; Colón-Ramos, Daniel A.; Wolf, Guy; Hirn, Matthew

doi:10.1109/bigdata47090.2019.9006013

Cited by 18 publications

(39 citation statements)

References 28 publications

(45 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Diffusion condensation is a dynamic process by which data points slowly and iteratively come together at a rate determined by the diffusion probabilities between them [12]. This iterative process is powerful as it reveals structure and groupings of the data at all levels of granularity for a given diffusion kernel bandwidth.…”

Section: Resultsmentioning

confidence: 99%

“…The first step involves computing a Markov diffusion operator from a dataset. This is done by first computing a distance matrix D between all data points, before converting this to an affinity matrix K by using an fixed bandwidth Gaussian kernel function as done previously [12]. K is then row normalized to obtain a diffusion operator between data points.…”

Section: Resultsmentioning

confidence: 99%

“…The original diffusion condensation algorithm [17] implements the coarse graining of the data in the ambient measurement space. However, condensing in this space can lead to “averaged” points that deviate from the intrinsic data manifold, especially in cases where the intrinsic manifold is very curved (Supplementary Figure 1A).…”

Section: Resultsmentioning

confidence: 99%

“…Based on this insight, we developed Multiscale PHATE, a method that can learn and visualize abstract cellular features and groupings of the data at all levels of granularity . Our algorithm is based on a dynamic process we have developed called diffusion condensation [12], which computes a manifold-intrinsic diffusion space on the original data before slowly condensing data points towards local centers of gravity to form natural, data-driven groupings across multiple granularities. Rather than forcing merges at each iteration, as done in most agglomerative hierarchical clustering methods, diffusion condensation allows cells to naturally come together over the course of successive condensation steps before merging points that fall within a threshold.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multiscale PHATE Exploration of SARS-CoV-2 Data Reveals Multimodal Signatures of Disease

Kuchroo

Huang

Wong

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

1SummaryThe biomedical community is producing increasingly high dimensional datasets, integrated from hundreds of patient samples, which current computational techniques struggle to explore. To uncover biological meaning from these complex datasets, we present an approach called Multiscale PHATE, which learns abstracted biological features from data that can be directly predictive of disease. Built on a continuous coarse graining process called diffusion condensation, Multiscale PHATE creates a tree of data granularities that can be cut at coarse levels for high level summarizations of data, as well as at fine levels for detailed representations on subsets. We apply Multiscale PHATE to study the immune response to COVID-19 in 54 million cells from 168 hospitalized patients. Through our analysis of patient samples, we identify CD16hi CD66blo neutrophil and IFNγ+GranzymeB+ Th17 cell responses enriched in patients who die. Further, we show that population groupings Multiscale PHATE discovers can be directly fed into a classifier to predict disease outcome. We also use Multiscale PHATE-derived features to construct two different manifolds of patients, one from abstracted flow cytometry features and another directly on patient clinical features, both associating immune subsets and clinical markers with outcome.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multiscale PHATE Exploration of SARS-CoV-2 Data Reveals Multimodal Signatures of Disease

Kuchroo

Huang

Wong

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In neuropils, densely packed neurons synaptically interconnect into precise circuit architecture 2,3 , yet the structural and developmental principles governing nanoscale precision in bundled neuropil assembly remain largely unknown [4][5][6] . Here we use diffusion condensation, a coarsegraining clustering algorithm 7 , to identify nested circuit structures within the C. elegans cerebral neuropil (called the nerve ring). We determine that the nerve ring neuropil is organized into four tightly bundled strata composed of related behavioral circuits.…”

Section: Discussionmentioning

confidence: 99%

Structural and developmental principles of neuropil assembly inC. elegans

Moyle

Barnes

Kuchroo

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

To visualize DC cluster relationships we developed C-PHATE, an extension of PHATE 30 capable of generating a 3D representation of the multigranular structures in the DC hierarchical tree ( Fig. 1a,b; see Methods). Quantitative comparisons of DC/C-PHATE results revealed similar clustering behaviors between the two separately analyzed nervering reconstructions (Adjusted Rand Index (ARI) of 0.7; Extended Data Fig. 1a,b), consistent with the long-accepted qualitative descriptions of the stereotyped structure for the C. elegans nerve ring 3,31 , and recent analyses of neurite adjacencies 22 . Examination of the cluster patterns through the iterations of DC revealed known cell-cell interactions and behavioral circuits ( Fig. 1b,c; Extended Data Fig. 2a,b; [32][33][34][35][36][37][38][39] ), consistent with behavioral and connectomic studies 10,19,20 . The multigranular outputs of the DC/C-PHATE results enabled understanding of the cell-cell interactions within the context of functional circuits, and the functional circuits within the context of the larger bundles of the neuropil.Comparisons of the modularity score, a measure of cluster robustness 40,41 , was highest when the majority of neurites were grouped into four super-clusters ( Fig. 1b; Extended Data Fig. 1a,b; Supplemental Video 1,2; see Methods). Overlaying our analysis of these four super-clusters (see Methods) onto the anatomy of the nerve ring revealed that they correspond to four distinct and tightly grouped bundles within the greater neuropil, circling the pharynx isthmus and stacked along the anterior-posterior axis of the animal. These clusters are herein called S1, S2, S3 and S4 for Stratum 1 etc. (Fig. 1c; Extended Data Fig. 1c-h; Supplemental Video 3 show details of these structures and single-cell identities). The stratified organization of the nerve ring neuropil, resolved here at single neurite level, is reminiscent of laminar organizations in the Drosophila nervous system 42,43 , and in the vertebrate retina and the brain cortex 44,45 .We observed that not all computationally clustered neurites followed simple bundled paths through the neuropil. In Strata 1 (S1), 32 anterior sensory neurons project perpendicular to the nerve ring bundle before making a 180º curl and returning to the anterior limits of the neuropil, where they terminate as synaptic endplates (Fig. 1d; Extended Data Fig. 3a-d; 3,46 ). These uniquely shaped S1 neurons looped at the borders of the S2/S3/S4 bundles. The anterior loops encase ~90% of S2, and the posterior loops encase ~84% of S3 and 100% of S4 ( Fig. 1d-g; Extended Data Fig. 3e-k ; SupplementalVideo 5; Supplemental Table 1). We found that the 32 axons form a six-fold symmetrical honeycomb structure along the neuropil's arc, creating a scaffold that encapsulates the different strata ( Fig. 1g; Extended Data Fig. 3e-h; Supplemental Video 4).The architecture of the neuropil reflects functional segregation of distinct sensory information streams and motor outputs.C. elegans neurons can be divided into three categories ...

show abstract