Efficient Compression of Genomic Sequences

Pratas, Diogo; Pinho, Armando J.; Ferreira, Paulo J. S. G.

doi:10.1109/dcc.2016.60

Cited by 44 publications

(35 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on this table, the best result, in terms of compression ratio, is not provided by a single approach. iDoComp [75] outperforms the other methods, including GReEn [60], GeCo [50] and GDC 2 [63] in three cases: compressing HS8 with reference HSCHM8, HS11 with reference PT11, and RICE5 with reference RICE7. Also, GDC 2 provides better results for the compression of HS11 with reference HSCHM11, as well as HSK16 with reference HS16.…”

Section: Resultsmentioning

confidence: 98%

See 1 more Smart Citation

A Survey on Data Compression Methods for Biological Sequences

2016

Self Cite

View full text Add to dashboard Cite

Abstract:The ever increasing growth of the production of high-throughput sequencing data poses a serious challenge to the storage, processing and transmission of these data. As frequently stated, it is a data deluge. Compression is essential to address this challenge-it reduces storage space and processing costs, along with speeding up data transmission. In this paper, we provide a comprehensive survey of existing compression approaches, that are specialized for biological data, including protein and DNA sequences. Also, we devote an important part of the paper to the approaches proposed for the compression of different file formats, such as FASTA, as well as FASTQ and SAM/BAM, which contain quality scores and metadata, in addition to the biological sequences. Then, we present a comparison of the performance of several methods, in terms of compression ratio, memory usage and compression/decompression time. Finally, we present some suggestions for future research on biological data compression.

show abstract

Section: Resultsmentioning

confidence: 98%

“…The GeCo algorithm [50], derived from [51,52], exploits a combination of context models of several orders for reference-free, as well as for reference-based genomic sequence compression. In this method, extended finite-context models (XFCMs), that are tolerant against substitution errors, are introduced.…”

Section: Reference-free Methodsmentioning

confidence: 99%

A Survey on Data Compression Methods for Biological Sequences

2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…We benchmark the performance of our neural network-based compressor DeepZip against Gzip, BSC [15], and some dataset specific compressors like GeCo [16] (for genomic data) and ZPAQ [17] (for text). For DeepZip we considered the three introduced probability predictor blocks FC, biGRU and LSTM-multi.…”

Section: Methodsmentioning

confidence: 99%

DeepZip: Lossless Data Compression Using Recurrent Neural Networks

Goyal

Tatwawadi

Chandak

et al. 2019

2019 Data Compression Conference (DCC)

View full text Add to dashboard Cite

Sequential data is being generated at an unprecedented pace in various forms, including text and genomic data. This creates the need for efficient compression mechanisms to enable better storage, transmission and processing of such data. To solve this problem, many of the existing compressors attempt to learn models for the data and perform predictionbased compression. Since neural networks are known as universal function approximators with the capability to learn arbitrarily complex mappings, and in practice show excellent performance in prediction tasks, we explore and devise methods to compress sequential data using neural network predictors. We combine recurrent neural network predictors with an arithmetic coder and losslessly compress a variety of synthetic, text and genomic datasets. The proposed compressor outperforms Gzip on the real datasets and achieves near-optimal compression for the synthetic datasets. The results also help understand why and where neural networks are good alternatives for traditional finite context models.

show abstract

“…DELIMINATE achieves better compression ratio than general-purpose compression methods although it uses almost the same compression time. In 2011, 2013, 2016, and 2019, respectively, Pinho et al published DNAEnc3 [31], MFCompress [32], GeCo [33], and GeCo2 [34] based on Markov models. DNAEnc3 partitions sequence to nonoverlapping blocks of fixed size, which are then encoded by the best one of the Markov models of different orders.…”

Section: Related Workmentioning

confidence: 99%

HRCM: An Efficient Hybrid Referential Compression Method for Genomic Big Data

Yao

et al. 2019

BioMed Research International

View full text Add to dashboard Cite

With the maturity of genome sequencing technology, huge amounts of sequence reads as well as assembled genomes are generating. With the explosive growth of genomic data, the storage and transmission of genomic data are facing enormous challenges. FASTA, as one of the main storage formats for genome sequences, is widely used in the Gene Bank because it eases sequence analysis and gene research and is easy to be read. Many compression methods for FASTA genome sequences have been proposed, but they still have room for improvement. For example, the compression ratio and speed are not so high and robust enough, and memory consumption is not ideal, etc. Therefore, it is of great significance to improve the efficiency, robustness, and practicability of genomic data compression to reduce the storage and transmission cost of genomic data further and promote the research and development of genomic technology. In this manuscript, a hybrid referential compression method (HRCM) for FASTA genome sequences is proposed. HRCM is a lossless compression method able to compress single sequence as well as large collections of sequences. It is implemented through three stages: sequence information extraction, sequence information matching, and sequence information encoding. A large number of experiments fully evaluated the performance of HRCM. Experimental verification shows that HRCM is superior to the best-known methods in genome batch compression. Moreover, HRCM memory consumption is relatively low and can be deployed on standard PCs.

show abstract

Efficient Compression of Genomic Sequences

Cited by 44 publications

References 15 publications

A Survey on Data Compression Methods for Biological Sequences

A Survey on Data Compression Methods for Biological Sequences

DeepZip: Lossless Data Compression Using Recurrent Neural Networks

HRCM: An Efficient Hybrid Referential Compression Method for Genomic Big Data

Contact Info

Product

Resources

About