Evaluation of methods for differential expression analysis on multi-group RNA-seq count data

Abstract

BackgroundRNA-seq is a powerful tool for measuring transcriptomes, especially for identifying differentially expressed genes or transcripts (DEGs) between sample groups. A number of methods have been developed for this task, and several evaluation studies have also been reported. However, those evaluations so far have been restricted to two-group comparisons. Accumulations of comparative studies for multi-group data are also desired.MethodsWe compare 12 pipelines available in nine R packages for detecting differential expressions (DE) from multi-group RNA-seq count data, focusing on three-group data with or without replicates. We evaluate those pipelines on the basis of both simulation data and real count data.ResultsAs a result, the pipelines in the TCC package performed comparably to or better than other pipelines under various simulation scenarios. TCC implements a multi-step normalization strategy (called DEGES) that internally uses functions provided by other representative packages (edgeR, DESeq2, and so on). We found considerably different numbers of identified DEGs (18.5 ~ 45.7 % of all genes) among the pipelines for the same real dataset but similar distributions of the classified expression patterns. We also found that DE results can roughly be estimated by the hierarchical dendrogram of sample clustering for the raw count data.ConclusionWe confirmed the DEGES-based pipelines implemented in TCC performed well in a three-group comparison as well as a two-group comparison. We recommend using the DEGES-based pipeline that internally uses edgeR (here called the EEE-E pipeline) for count data with replicates (especially for small sample sizes). For data without replicates, the DEGES-based pipeline with DESeq2 (called SSS-S) can be recommended.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0794-7) contains supplementary material, which is available to authorized users.

A powerful and flexible tool for rapid and accurate differential expression and alternative splicing analysis of RNA-seq data for biologists

“…Limma voom (Law et al, 2014) was chosen as the engine for the DE analysis (DE, DAS and DTU) for four reasons. Firstly, from different studies, limma is consistently one of the best performing methods for RNA-seq analysis and has a good control of FDR (Pimentel et al, 2017;Rapaport et al, 2013;Tang et al, 2015).…”

Data-based RNA-seq Simulations by Binomial Thinning

“…Though the two-group model is perhaps the most common scenario in differential expression analysis, our method also allows for arbitrary design matrices. Such design matrices have applications in many types of expression experiments [Smyth, 2004, McCarthy et al, 2012, Van De Wiel et al, 2012, Tang et al, 2015, and so the ability to simulate arbitrary designs gives researchers another tool to evaluate their methods in more complicated scenarios.…”

“…Real data often exhibit unwanted variation beyond that assumed by a model . Theoretical distributional assumptions are also difficult to verify, and are sometimes mired in controversy [Svensson, 2019].…”

dearseq: a variance component score test for RNA-Seq differential analysis that effectively controls the false discovery rate

“…When comparing DEA methods, the evaluation of their empirical FDR with respect to the targeted (nominal) level is often overlooked [5,6,7,8,9,10]. Nonetheless, some issues with inflated FDR in DEA have been previously reported in the literature [11,12,13,14,15], but those warnings have made little apparent impact on DEA practices.…”