Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

Background The increasing production of genomic data has led to an intensified need for models that can cope efficiently with the lossless compression of DNA sequences. Important applications include long-term storage and compression-based data analysis. In the literature, only a few recent articles propose the use of neural networks for DNA sequence compression. However, they fall short when compared with specific DNA compression tools, such as GeCo2. This limitation is due to the absence of models specifically designed for DNA sequences. In this work, we combine the power of neural networks with specific DNA models. For this purpose, we created GeCo3, a new genomic sequence compressor that uses neural networks for mixing multiple context and substitution-tolerant context models. Findings We benchmark GeCo3 as a reference-free DNA compressor in 5 datasets, including a balanced and comprehensive dataset of DNA sequences, the Y-chromosome and human mitogenome, 2 compilations of archaeal and virus genomes, 4 whole genomes, and 2 collections of FASTQ data of a human virome and ancient DNA. GeCo3 achieves a solid improvement in compression over the previous version (GeCo2) of $2.4\%$, $7.1\%$, $6.1\%$, $5.8\%$, and $6.0\%$, respectively. To test its performance as a reference-based DNA compressor, we benchmark GeCo3 in 4 datasets constituted by the pairwise compression of the chromosomes of the genomes of several primates. GeCo3 improves the compression in $12.4\%$, $11.7\%$, $10.8\%$, and $10.1\%$ over the state of the art. The cost of this compression improvement is some additional computational time (1.7–3 times slower than GeCo2). The RAM use is constant, and the tool scales efficiently, independently of the sequence size. Overall, these values outperform the state of the art. Conclusions GeCo3 is a genomic sequence compressor with a neural network mixing approach that provides additional gains over top specific genomic compressors. The proposed mixing method is portable, requiring only the probabilities of the models as inputs, providing easy adaptation to other data compressors or compression-based data analysis tools. GeCo3 is released under GPLv3 and is available for free download at https://github.com/cobilab/geco3.

show abstract

“…DS4 : highly repetitive DNA with the human Y-chromosome (HoSaY) and a human mitogenome collection (Mito) (proposed in [ 90 ]);…”

Section: Resultsmentioning

confidence: 99%

Efficient DNA sequence compression with neural networks

2020

View full text Add to dashboard Cite

show abstract

“…The code is not highly optimized in terms of speed and memory usage in both compression and decompression, but is sufficient as the main goal of this version of implementation is to proof the concept of the OST algorithm and conduct a trial testing and investigation. A set of genomes as listed in Table 2 and 3, which is the same dataset used in 110 , were used for testing and comparing the results of OST-DNA with the results of the tools in Table 1.…”

Section: Resultsmentioning

confidence: 99%

A Novel Lossless Encoding Algorithm for Data Compression - Genomics Data as an Exemplar

Al-okaily

Tbakhi

2020

Preprint

View full text Add to dashboard Cite

Data compression is a fundamental problem in the fields of computer science, information theory, and coding theory. The need for compressing data is to reduce the size of the data so that the storage and the transmission of them become more efficient. Motivated from resolving the compression of DNA data, we introduce a novel encoding algorithm that works for any textual data including DNA data. Moreover, the design of this algorithm paves a novel approach so that researchers can build up on and resolve better the compression problem of DNA or textual data.

show abstract

“…The DENV nr dataset ( Section 2.4 ) was used for performance comparison analysis between ITERmin, a re-implementation herein of the earlier Khan algorithm (KA) [ 20 , 21 ], and UNIQmin. Additionally, a literature search showed that several protein compressors have been developed [ 42 , 43 ], which, however, mainly focus on the reduction of sequence file size for storage purposes. Nonetheless, a comparison was performed against the existing compressors to evaluate the compression ability of UNIQmin.…”

Section: Methodsmentioning

confidence: 99%

“…An HS dataset was used because it was the only one that allowed both direct and an approximate indirect (with 2019 HS dataset) comparison between the compressors. Direct comparison of UNIQmin was only carried out with Gzip ( , accessed on 3 March 2020), a widely used compressor that balances speed and compression [ 43 ], and AC, which typically showcases the best compression [ 42 ]. The direct comparison between UNIQmin, Gzip, and AC was made by use of the 2020 HS dataset (HS 2020 ) and 2018 all reported viral sequence dataset (“All viruses”), retrieved from the NCBI Entrez Protein Database by use of the txid “10239” (as of November 2018), while indirect comparisons were made with other tools using earlier HS datasets (HS 2019 ) and viral datasets of Acanthamoeba polyphaga (AP 2019 ) and Enterococcus phage (EP 2019 ) [ 42 ].…”

Section: Methodsmentioning

confidence: 99%

An Alignment-Independent Approach for the Study of Viral Sequence Diversity at Any Given Rank of Taxonomy Lineage

Lim

Ban

et al. 2021

Biology

View full text Add to dashboard Cite

The study of viral diversity is imperative in understanding sequence change and its implications for intervention strategies. The widely used alignment-dependent approaches to study viral diversity are limited in their utility as sequence dissimilarity increases, particularly when expanded to the genus or higher ranks of viral species lineage. Herein, we present an alignment-independent algorithm, implemented as a tool, UNIQmin, to determine the effective viral sequence diversity at any rank of the viral taxonomy lineage. This is done by performing an exhaustive search to generate the minimal set of sequences for a given viral non-redundant sequence dataset. The minimal set is comprised of the smallest possible number of unique sequences required to capture the diversity inherent in the complete set of overlapping k-mers encoded by all the unique sequences in the given dataset. Such dataset compression is possible through the removal of unique sequences, whose entire repertoire of overlapping k-mers can be represented by other sequences, thus rendering them redundant to the collective pool of sequence diversity. A significant reduction, namely ~44%, ~45%, and ~53%, was observed for all reported unique sequences of species Dengue virus, genus Flavivirus, and family Flaviviridae, respectively, while still capturing the entire repertoire of nonamer (9-mer) viral peptidome diversity present in the initial input dataset. The algorithm is scalable for big data as it was applied to ~2.2 million non-redundant sequences of all reported viruses. UNIQmin is open source and publicly available on GitHub. The concept of a minimal set is generic and, thus, potentially applicable to other pathogenic microorganisms of non-viral origin, such as bacteria.

show abstract

Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

Cited by 27 publications

References 47 publications

Efficient DNA sequence compression with neural networks

Efficient DNA sequence compression with neural networks

A Novel Lossless Encoding Algorithm for Data Compression - Genomics Data as an Exemplar

An Alignment-Independent Approach for the Study of Viral Sequence Diversity at Any Given Rank of Taxonomy Lineage

Contact Info

Product

Resources

About