DNABIT Compress-Genome Compression Algorithm

Technological advancements in high-throughput sequencing have lead to a tremendous increase in the amount of genomic data produced. With the cost being down to 2,000 USD for a single human genome, sequencing dozens of individuals is a task that is feasible even for smaller project or organizations already today. However, generating the sequence is only one issue; another one is storing, managing, and analyzing it. These tasks become more and more challenging due to the sheer size of the data sets and are increasingly considered to be the most severe bottlenecks in larger genome projects. One possible countermeasure is to compress the data; compression reduces costs in terms of requiring less hard disk storage and in terms of requiring less bandwidth if data is shipped to large compute clusters for parallel analysis. Accordingly, sequence compression has recently attracted much interest in the scientific community. In this paper, we explain the different basic techniques for sequence compression, point to distinctions between different compression tasks (e.g., genome versus read compression), and present a comparison of current approaches and tools. To further stimulate progress in genome compression research, we also identify key challenges for future systems.Keywords: genome compression, read compression, survey Key messages of the article:• Overview of trends in genome compression: bit manipulation, dictionarybased, statistical, and referential compression schemes• Wide ranges of compression performance (compression ratio, compression time, memory requirements)• Comparing compression schemes for evaluation purposes is difficult• Many minor improvements in recent contributions

show abstract

“…[35] describes a particular run-length encoding scheme. Finally, [36] focuses on the analysis of how to store repeats with variable-size codes.…”

Section: Naive Bit Manipulation Algorithmsmentioning

confidence: 99%

Trends in Genome Compression

Wandelt

Bux

Leser

2014

CBIO

View full text Add to dashboard Cite

show abstract

“…Existing short-read compression approaches generally fall into categories: reference-based schemes (Campagne et al, 2013; Fritz et al, 2011; Li et al, 2014) attempt to compress reads by aligning them to one or more known reference sequences and recording edits between the read and its mapped location in the reference. De novo compression schemes (Adjeroh et al, 2002; Bhola et al, 2011; Bonfield and Mahoney, 2013; Brandon et al, 2009; Burriesci et al, 2012; Cox et al, 2012; Deorowicz and Grabowski, 2011; Hach et al, 2012; Jones et al, 2012; Kozanitis et al, 2011; Popitsch and von Haeseler, 2013; Rajarajeswari and Apparao, 2011; Tembe et al, 2010) attempt to compress without appeal to a reference. SCALCE (Hach et al, 2012) is one of the most effective, and works by reordering reads within the FASTA file to boost the compression of general purpose compressors.…”

Section: Introductionmentioning

confidence: 99%

Compression of short-read sequences using path encoding

Kingsford

Patro

2014

Preprint

View full text Add to dashboard Cite

Storing, transmitting, and archiving the amount of data produced by next generation sequencing is becoming a significant computational burden. For example, large-scale RNA-seq meta-analyses may now routinely process tens of terabytes of sequence. We present here an approach to biological sequence compression that reduces the difficulty associated with managing the data produced by large-scale transcriptome sequencing. Our approach offers a new direction by sitting between pure reference-based compression and reference-free compression and combines much of the benefit of reference-based approaches with the flexibility of de novo encoding. Our method, called path encoding, draws a connection between storing paths in de Bruijn graphs — a common task in genome assembly — and context-dependent arithmetic coding. Supporting this method is a system, called a bit tree, to compactly store sets of kmers that is of independent interest. Using these techniques, we are able to encode RNA-seq reads using 3% – 11% of the space of the sequence in raw FASTA files, which is on average more than 34% smaller than recent competing approaches. We also show that even if the reference is very poorly matched to the reads that are being encoded, good compression can still be achieved.

show abstract

“…DNABIT (Rajarajeswari and Apparao, 2011) has two phases, first even bit technique which assigns two bits for every nucleotide of non-repeat regions; second phase is odd bit technique which assigns 3, 5, 7 or 9 bits based on the size of repeat regions.…”

Section: Related Workmentioning

confidence: 99%

“…General purpose compression algorithms expand the sequences rather than compressing (Rajarajeswari and Apparao, 2011), so they cannot achieve the same compression ratio as specialized DNA sequences compression algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Deoxyribonucleic Acid Compression Algorithm Using Auto-Regression and Swarm Intelligence

Aly¹,

Yousif²,

Zohdy³

2013

Journal of Computer Science

View full text Add to dashboard Cite

DNA compression challenge has become a major task for many researchers as a result of exponential increase of produced DNA sequences in gene databases; in this research we attempt to solve the DNA compression challenge by developing a lossless compression algorithm. The proposed algorithm works in horizontal mode using a substitutional-statistical technique which is based on Auto Regression modeling (AR), the model parameters are determined using Particle Swarm Optimization (PSO). This algorithm is called Swarm Auto-Regression DNA Compression (SARDNAComp). SARDNAComp aims to reach higher compression ratio which make its application beneficial for both practical and functional aspects due to reduction of storage, retrieval, transmission costs and inferring structure and function of sequences from compression, SARDNAComp is tested on eleven benchmark DNA sequences and compared to current algorithms of DNA compression, the results showed that (SARDNAComp) outperform these algorithms.

show abstract

DNABIT Compress-Genome Compression Algorithm

Cited by 35 publications

References 17 publications

Trends in Genome Compression

Trends in Genome Compression

Compression of short-read sequences using path encoding

A Deoxyribonucleic Acid Compression Algorithm Using Auto-Regression and Swarm Intelligence

Contact Info

Product

Resources

About