Confronting false discoveries in single-cell differential expression

Abstract

Differential expression analysis in single-cell transcriptomics enables the dissection of cell-type-specific responses to perturbations such as disease, trauma, or experimental manipulations. While many statistical methods are available to identify differentially expressed genes, the principles that distinguish these methods and their performance remain unclear. Here, we show that the relative performance of these methods is contingent on their ability to account for variation between biological replicates. Methods that ignore this inevitable variation are biased and prone to false discoveries. Indeed, the most widely used methods can discover hundreds of differentially expressed genes in the absence of biological differences. To exemplify these principles, we exposed true and false discoveries of differentially expressed genes in the injured mouse spinal cord.

Robust Transcriptional Profiling and Identification of Differentially Expressed Genes With Low Input RNA Sequencing of Adult Hippocampal Neural Stem and Progenitor Populations

“…Furthermore, our own investigation of false positive rate in scRNAseq, when cells were randomly assigned to two groups, also yielded no false DEGs, consistent with findings in Soneson and Robinson (2018), who performed similar analyses over multiple scRNAseq datasets and found low (sometimes no) false discoveries when using the same statistical analysis approach that we used (Wilcoxon). In contrast to these findings, Squair et al (2021) recently published findings suggesting that divergence of scRNAseq differential expression analysis from that of matched bulk RNAseq represented false positives in scRNAseq. This conclusion was mostly based on simulations where they randomly assigned pseudo-replicates or real replicates to treatment groups.…”

“…Particularly when replicateto-replicate variation was high, large numbers of false positives were found in this analysis. Squair et al (2021) also used an RNAscope assay to attempt to confirm scRNAseq-derived DEGs in a model of spinal cord injury and found low replicability there. RNAscope is commonly used to confirm scRNAseq findings so it is unclear why replicability is typically reported in other studies but not found in Squair et al (2021).…”

“…However, the low degree of overlap in the individual DEGs identified by the 2 platforms indicates that some form of bias affected the type of genes identified by each platform. One recent study similarly found divergence in DEG detection between single cell and bulk approaches but concluded that this represented error on the part of the single cell data (Squair et al, 2021). A core piece of evidence leading to this conclusion was a high rate of false positive DEGs generated from single cell data when treatment versus control conditions were assigned randomly.…”

“…A small handful of studies has addressed robustness of different statistical analysis approaches for DEG identification, but some have contradictory findings (Ziegenhain et al, 2017;Soneson and Robinson, 2018;Wang et al, 2019;Mou et al, 2020;Squair et al, 2021). For example, Squair et al (2021) suggest that scRNAseq DEGs are rife with false positives while Soneson and Robinson (2018) show that the most common statistical tests are quite resistant to false positives in scRNAseq DEG analysis. Most studies of this nature are retrospective-taking advantage of samples not originally processed for the purpose of identifying overlap and error in methods.…”

Comparative transcriptional analysis of the satellite glial cell injury response

“…Another likely cause for the observed variability is that we were limited to performing the differential expression analyses on a cell-by-cell basis, an approach which lacks power and gives rise to a higher frequency of false positives. If we had had more biological replicates, we could have performed a pseudo-bulk analysis which might have shed further light on the common responses of SGCs to different nerve injuries (Crowell et al, 2020;Squair et al, 2021).…”