BioSeqZip: a collapser of NGS redundant reads for the optimization of sequence analysis

Abstract

Motivation High-throughput next-generation sequencing can generate huge sequence files, whose analysis requires alignment algorithms that are typically very demanding in terms of memory and computational resources. This is a significant issue, especially for machines with limited hardware capabilities. As the redundancy of the sequences typically increases with coverage, collapsing such files into compact sets of non-redundant reads has the 2-fold advantage of reducing file size and speeding-up the alignment, avoiding to map the same sequence multiple times. Method BioSeqZip generates compact and sorted lists of alignment-ready non-redundant sequences, keeping track of their occurrences in the raw files as well as of their quality score information. By exploiting a memory-constrained external sorting algorithm, it can be executed on either single- or multi-sample datasets even on computers with medium computational capabilities. On request, it can even re-expand the compacted files to their original state. Results Our extensive experiments on RNA-Seq data show that BioSeqZip considerably brings down the computational costs of a standard sequence analysis pipeline, with particular benefits for the alignment procedures that typically have the highest requirements in terms of memory and execution time. In our tests, BioSeqZip was able to compact 2.7 billion of reads into 963 million of unique tags reducing the size of sequence files up to 70% and speeding-up the alignment by 50% at least. Availability and implementation BioSeqZip is available at https://github.com/bioinformatics-polito/BioSeqZip. Supplementary information Supplementary data are available at Bioinformatics online.

Immunological pathways of macrophage response to Brucella ovis infection

“…Compared with first-generation sequencing technology, high-throughput sequencing technology is a milestone in the field of genomics because of lower costs and better opportunity to identify DEGs for further studying the pathogenesis of diseases. 39 – 41 Brucella can be transmitted through contaminated placenta and aerosol, and can enter the host through gastrointestinal and respiratory mucosae. The mucosal immune response is the host’s main defense host against invasive Brucella .…”

Complexity analysis of algorithms: A case study on bioinformatics tools

“…Tools and data source: The tools selected for this analysis were MarDRe (Expósito et al, 2017), ParDRe, FastUniq (Xu et al, 2012), NGS Reads Treatment (Gaia et al, 2019), and BioSeqZip (Urgese et al, 2020). They were chosen because they are tools capable of manipulating platform-independent NGS data and are freely available to the scientific community.…”

Fast-HBR: Fast hash based duplicate read remover

“…BioSeqZip [10] consume almost the same memory amount in all datasets because of its memory limit control. Therefore, it has a smaller memory footprint than Fast-HBR in all datasets except SRR10315305.…”

“…Some examples of de novo tools are CD-HIT [ 6 ], FastUniq [ 7 ] and Fulcrum [ 8 ]. Available de novo tools include NGS Reads Treatment [ 9 ], Nubeam-dedup [ 5 ], BioSeqZip [ 10 ] and Minirmd [ 11 ]. NGS Reads Treatment [ 9 ] is a hash-based tool that uses Cuckoo Filter [ 12 ] which is a probabilistic data structure.…”

“…The authors elsewhere [ 5 ] developed the Nubeam-dedup tool that uses Nubeam [ 13 ] to represent each read as a number by calculating a product of matrices that represent nucleotides in the read. The BioSeqZip [ 10 ] tool starts by splitting the reads into small chunks, and then it sorts them alphabetically with memory limiting feature having long processing time. Minirmd [ 11 ] with the help of k-minimizer [ 14 ] clusters the reads into groups, where each group will contain reads that have the same k-minimizer in the same position.…”