New evaluation methods of read mapping by 17 aligners on simulated and empirical NGS data: an updated comparison of DNA- and RNA-Seq data from Illumina and Ion Torrent technologies

Abstract

During the last (15) years, improved omics sequencing technologies have expanded the scale and resolution of various biological applications, generating high-throughput datasets that require carefully chosen software tools to be processed. Therefore, following the sequencing development, bioinformatics researchers have been challenged to implement alignment algorithms for next-generation sequencing reads. However, nowadays selection of aligners based on genome characteristics is poorly studied, so our benchmarking study extended the “state of art” comparing 17 different aligners. The chosen tools were assessed on empirical human DNA- and RNA-Seq data, as well as on simulated datasets in human and mouse, evaluating a set of parameters previously not considered in such kind of benchmarks. As expected, we found that each tool was the best in specific conditions. For Ion Torrent single-end RNA-Seq samples, the most suitable aligners were CLC and BWA-MEM, which reached the best results in terms of efficiency, accuracy, duplication rate, saturation profile and running time. About Illumina paired-end osteomyelitis transcriptomics data, instead, the best performer algorithm, together with the already cited CLC, resulted Novoalign, which excelled in accuracy and saturation analyses. Segemehl and DNASTAR performed the best on both DNA-Seq data, with Segemehl particularly suitable for exome data. In conclusion, our study could guide users in the selection of a suitable aligner based on genome and transcriptome characteristics. However, several other aspects, emerged from our work, should be considered in the evolution of alignment research area, such as the involvement of artificial intelligence to support cloud computing and mapping to multiple genomes. Supplementary Information The online version contains supplementary material available at 10.1007/s00521-021-06188-z.

A Comparative Study of RNA-Seq Aligners Reveals Novoalign’s Default Setting as an Optimal Setting for the Alignment of HeLa RNA-Seq Reads

“…However, the recall values between these two aligners were equally high (>0.92), except for the Novoalign Tweak 2 parameter setting that obtained an extremely low recall value of 0.79419. A previous study by Donato et al (2021) compared 17 aligners on simulated and empirical NGS data, and the findings revealed that Novoalign showed the highest accuracy in all alignments.…”

“…In contrast to Subread, the default setting of Novoalign required more than 2 hours of runtime to obtain a comparable precision and recall. In addition, Novoalign had the longest computational time of reads mapping, probably due to its predispositions toward other parameters (Donato et al, 2021).…”

“…Subread was found to be the most vulnerable to indels among these aligners. The main limitation in the gapped alignment-based indels detection method is the need for indels that are entirely contained within a read and correctly detected during the initial read mapping step (Donato et al, 2021;Li et al, 2009). In the case of small indel detection, the supporting reads are frequently presented with too few bases that can match with the reference genome, or the reads may contain only one end that can map correctly to the reference genome.…”

“…In the case of small indel detection, the supporting reads are frequently presented with too few bases that can match with the reference genome, or the reads may contain only one end that can map correctly to the reference genome. However, the remaining bases following the indel may be trimmed or soft-clipped by the NGS aligner (Donato et al, 2021;Landman et al, 2014).…”

“…The authors revealed that the aligners' performances varied by genome complexity. Unfortunately, although many benchmarking analyses had been carried out in guiding the users in choosing the best aligners, the problem is still plaguing the bioinformatics communities, while other solutions have not been derived (Donato et al, 2021;Grytten et al, 2020;Jain et al, 2020;Schilbert et al, 2020;Thankaswamy-Kosalai et al, 2017).…”