A comprehensive comparison of supervised and unsupervised methods for cell type identification in single-cell RNA-seq

Abstract

The cell type identification is among the most important tasks in single-cell RNA-sequencing (scRNA-seq) analysis. Many in silico methods have been developed and can be roughly categorized as either supervised or unsupervised. In this study, we investigated the performances of 8 supervised and 10 unsupervised cell type identification methods using 14 public scRNA-seq datasets of different tissues, sequencing protocols and species. We investigated the impacts of a number of factors, including total amount of cells, number of cell types, sequencing depth, batch effects, reference bias, cell population imbalance, unknown/novel cell type, and computational efficiency and scalability. Instead of merely comparing individual methods, we focused on factors’ impacts on the general category of supervised and unsupervised methods. We found that in most scenarios, the supervised methods outperformed the unsupervised methods, except for the identification of unknown cell types. This is particularly true when the supervised methods use a reference dataset with high informational sufficiency, low complexity and high similarity to the query dataset. However, such outperformance could be undermined by some undesired dataset properties investigated in this study, which lead to uninformative and biased reference datasets. In these scenarios, unsupervised methods could be comparable to supervised methods. Our study not only explained the cell typing methods’ behaviors under different experimental settings but also provided a general guideline for the choice of method according to the scientific goal and dataset properties. Finally, our evaluation workflow is implemented as a modularized R pipeline that allows future evaluation of new methods. Availability: All the source codes are available at https://github.com/xsun28/scRNAIdent.

IBRAP: Integrated Benchmarking Single-cell RNA-sequencing Analytical Pipeline

“…It is worth noting that supervised methods are heavily constrained by the quality and the bandwidth of cell types in the supplied reference. If the reference is low in resolution, it would be very difficult to identify rare and novel populations as previously described [7].…”

“…It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in standard cell annotation involves unsupervised clustering followed by differential expression and cluster annotation using top differentially expressed genes for each cluster. Cell annotation using reference-based maps (i.e., supervised cell annotation) has, however, been shown to cluster imbalanced cellular proportions (with rare cell types) more effectively compared to unsupervised methods [7]. As both unsupervised and supervised clustering functions were included in IBRAP, we wanted to investigate how the performance varies between the two in cell clustering and annotation, using the 8 pancreatic samples from the individual analysis all with ground truth labels.…”

“…normalisation, integration, clustering) or as pipelines as a 4 whole. Conclusively, there was no single approach able to gain the best results across all sample scenarios which leads to multiple tool recommendations or case-by-case recommendations [2,[4][5][6][7][8][9][10][11][12]. Since tools are not integrated, custom pipeline coding is required which is tedious with inevitable room for errors.…”

How does data structure impact cell-cell similarity? Evaluating the influence of structural properties on proximity metric performance in single cell RNA-seq data

“…The transcriptional profiles defining the current state of individual cells can be studied at high-resolution to identify signature genes, and patterns of expression which denote specific cellular processes [1], states [2], and types [3]. Accordingly, unsupervised clustering algorithms have become a popular approach in scRNA-seq for the identification of cell-types in an unbiased manner [4][5][6]. These algorithms partition cells into distinct clusters on the basis of cell-cell distances using a proximity metric, such as Euclidean distance.…”

D3K: The Dissimilarity-Density-Dynamic Radius K-means Clustering Algorithm for scRNA-Seq Data

“…In order to obtain the information needed by people, bioinformatics emerges as The Times require (Li and Wong, 2019;Liu et al, 2021). It is an interdisciplinary subject composed of life science and computer science, which can dig out the biological significance contained in the chaotic biological data (Sun et al, 2022). Transcriptome is an important research field in bioinformatics, which can study gene function and gene structure from an overall level, and reveal specific biological processes and molecular mechanisms in the process of disease occurrence (Qi et al, 2021;Tang et al, 2020).…”