Accelerated Hierarchical Density Based Clustering

Abstract

We present an accelerated algorithm for hierarchical density based clustering. Our new algorithm improves upon HDBSCAN*, which itself provided a significant qualitative improvement over the popular DBSCAN algorithm. The accelerated HDBSCAN* algorithm provides comparable performance to DBSCAN, while supporting variable density clusters, and eliminating the need for the difficult to tune distance scale parameter . This makes accelerated HDBSCAN* the default choice for density based clustering. arXiv:1705.07321v2 [stat.ML] 23 May 2017 1 See https://github.com/lmcinnes/hdbscan_paper/blob/master/Qualitative% 20clustering%20results.ipynb for code used to generate these plots

Biologically indeterminate yet ordered promiscuous gene expression in single medullary thymic epithelial cells

“…The gene-frequencyby-inverse cell-frequency matrix was further reduced to a gene-by-context matrix by using a tdistributed stochastic neighbour embedding (t-SNE) (Maaten & Hinton, 2008) to reduce the cosine distance of the first 50 eigenvectors of the gene-frequency-by-inverse cell-frequency matrix (acquired using singular value decomposition). This reduced dimensionality gene-bycontext matrix was then clustered using HDBSCAN (McInnes & Healy, 2017) to spatially select clusters based on density in the reduced dimensionality representation. This has the benefit of identifying the sets of genes that are repeatedly observed together in the same context (subsets of cells), while simultaneously attenuating the signal from frequently expressed genes unless accompanied by a drastic change in expression level.…”

A machine perspective of atomic defects in scanning transmission electron microscopy

“…For structure identification problems in the third stage, the central task is to automatically combine feature vectors of the same kind into a single group. Cluster analysis encompasses different algorithms involving K‐means, hierarchical clustering or density‐based spatial clustering of applications with noise (DBSCAN) . The performance of each approach relies on the quality of the input feature vectors retrieved in the second stage.…”

“…Cluster analysis encompasses different algorithms involving K-means, 107 hierarchical clustering or density-based spatial clustering of applications with noise (DBSCAN). [108][109][110] The performance of each approach relies on the quality of the input feature vectors retrieved in the second stage. In the following subsection, we will discuss the advantages and disadvantages of various clustering methods employed in different scenarios.…”

Stable Components and Layers

“…Example 3. The "hierarchical" version HDBSCAN* [1], [4] of DBSCAN* is the production (and interpretation) of a tree that is associated to the string of functions π 0 L s0,k (X) → π 0 L s1,k (X) → · · · → π 0 L sp,k (X) = *…”

“…Scoring appears as an analytic device in statistical approaches to clustering -see [4], for example. An alternative combinatorial method of scoring is presented in the third section of this paper.…”