ClusterDE: a post-clustering differential expression (DE) method robust to false-positive inflation caused by double dipping

Abstract

In typical single-cell RNA-seq (scRNA-seq) data analysis, a clustering algorithm is applied to find putative cell types as clusters, and then a statistical differential expression (DE) test is used to identify the differentially expressed (DE) genes between the cell clusters. However, this common procedure uses the same data twice, an issue known as "double dipping": the same data is used to define both cell clusters and DE genes, leading to false-positive DE genes even when the cell clusters are spurious. To overcome this challenge, we propose ClusterDE, a post-clustering DE test for controlling the false discovery rate (FDR) of identified DE genes regardless of clustering quality. The core idea of ClusterDE is to generate real-data-based synthetic null data with only one cluster, as a counterfactual in contrast to the real data, for evaluating the whole procedure of clustering followed by a DE test. Using comprehensive simulation and real data analysis, we show that ClusterDE has not only solid FDR control but also the ability to find cell-type marker genes that are biologically meaningful. ClusterDE is fast, transparent, and adaptive to a wide range of clustering algorithms and DE tests. Besides scRNA-seq data, ClusterDE is generally applicable to post-clustering DE analysis, including single-cell multi-omics data analysis.

A knockoff calibration method to avoid over-clustering in single-cell RNA-sequencing

“…Differences between callback and ClusterDE. Both callback and ClusterDE [6] use synthetic null variables and the knockoff filter. The key distinction between these methods is that ClusterDE takes given cell clusters and computes knockoff data to calibrate statistical null hypothesis tests between those clusters, while callback computes knockoff data on the full dataset first and uses the augmented data matrix as input to the clustering algorithm in order to calibrate the choice of clusters.…”

“…The most commonly used software packages, such as [4] and [5], perform these two steps on the same dataset. This double use of data is often referred to as “circular analysis” or “double-dipping,” and is known to result in highly inflated P -values, even in the null case when gene expression is identically distributed and there are no true groupings that distinguish cell populations [6, 7]. Due to the miscalibrated test statistics produced by circular analyses, it is challenging to assess whether the genes found to be differentially expressed between two putative cell groups are “real” or solely identified due to chance based on the way that the cells are being partitioned by the clustering algorithm that is being used.…”

“…in the null case when gene expression is identically distributed and there are no true groupings that distinguish cell populations [6,7]. Due to the miscalibrated test statistics produced by circular analyses, it is challenging to assess whether the genes found to be differentially expressed between two putative cell groups are "real" or solely identified due to chance based on the way that the cells are being partitioned by the clustering algorithm that is being used.…”

Mapping the gene space at single-cell resolution with gene signal pattern analysis

“…We highly rank genes that are close to zero expression in all other cell types and close to uniform expression in the cell type of interest. Additionally, conventional methods for detecting differential gene expression between clusters often return inflated p-values because of the double use of gene expression data, first to partition the data into clusters and then to define significance statistics along the same partitions [50]. Consequently, filtering based on p-value alone results in an increased rate of false positives, and some pipelines have turned to gene rankings instead of cutoffs [60].…”

“…We highlight the importance of genes that are highly localized on the cellular manifold versus genes that are diffusely expressed, hypothesizing that such genes are beneficial in characterizing different areas of the manifold informatively. We also show that differential localization has several advantages compared to standard cluster-based differential expression pipelines, including reduced false positives [50] and the identification of genes with patterns of localization that do not conform to cluster boundaries [27].…”