Performance comparison of sequential and parallel compression applications for DNA raw data

Guerra, Aníbal; Lotero, Jaime; Isaza, Sebastian

doi:10.1007/s11227-016-1753-4

Cited by 9 publications

(11 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…39 Several lossless non-referential compressors for FASTQ have been released since then 16,23,27–35,39–41 and most of them have been reviewed and tested in detail. 9,17,25,42 In our previous tests, top performers achieved compression ratios in the range between 4:1 and 8:1, which is still below from what a referential compressor could theoretically achieve. Also, restrictions related to input data features (file size, read size, technology of the sequencing machine), excessive runtime or low compression ratios, have limited the usage and effectiveness of non-referential compressors.…”

Section: Related Workmentioning

confidence: 66%

See 1 more Smart Citation

Tackling the Challenges of FASTQ Referential Compression

Guerra

Lotero

Aedo

et al. 2019

Bioinform Biol Insights

Self Cite

View full text Add to dashboard Cite

The exponential growth of genomic data has recently motivated the development of compression algorithms to tackle the storage capacity limitations in bioinformatics centers. Referential compressors could theoretically achieve a much higher compression than their non-referential counterparts; however, the latest tools have not been able to harness such potential yet. To reach such goal, an efficient encoding model to represent the differences between the input and the reference is needed. In this article, we introduce a novel approach for referential compression of FASTQ files. The core of our compression scheme consists of a referential compressor based on the combination of local alignments with binary encoding optimized for long reads. Here we present the algorithms and performance tests developed for our reads compression algorithm, named UdeACompress. Our compressor achieved the best results when compressing long reads and competitive compression ratios for shorter reads when compared to the best programs in the state of the art. As an added value, it also showed reasonable execution times and memory consumption, in comparison with similar tools.

show abstract

Section: Related Workmentioning

confidence: 66%

“…These types of tools have seen widespread use for compressing biological sequences 22,23 due to their compatibility, robustness, and ease of use, in spite of certain performance limitations. 24,25…”

Section: Introductionmentioning

confidence: 99%

Tackling the Challenges of FASTQ Referential Compression

Guerra

Lotero

Aedo

et al. 2019

Bioinform Biol Insights

Self Cite

View full text Add to dashboard Cite

show abstract

“…Evaluating the effectiveness of high-throughput sequencing data compression tools has gained a lot of interest in the last few years [1, 13–15]. Comparative reviews of prominent general-purpose as well as DNA-specific compression algorithms show that DNA compression algorithms tend to compress DNA sequences much better than general-purpose compression algorithms [1, 4].…”

Section: Discussionmentioning

confidence: 99%

“…read identifier and read sequence) are compressed using MFCompress after the identifier stream is pre-processed to comply with the format restrictions of MFCompress. The third stream is discarded during compression as it contains a ’+’ symbol followed by an optional comment similar to identifier field which can be regenerated later at the time of decompression [13]. This is similar to all available tools including those used for comparison in this study.…”

Section: Methodsmentioning

confidence: 99%

MZPAQ: a FASTQ data compression tool

Allali

Arshad

2019

Source Code Biol Med

View full text Add to dashboard Cite

Background Due to the technological progress in Next Generation Sequencing (NGS), the amount of genomic data that is produced daily has seen a tremendous increase. This increase has shifted the bottleneck of genomic projects from sequencing to computation and specifically storing, managing and analyzing the large amount of NGS data. Compression tools can reduce the physical storage used to save large amount of genomic data as well as the bandwidth used to transfer this data. Recently, DNA sequence compression has gained much attention among researchers. Results In this paper, we study different techniques and algorithms used to compress genomic data. Most of these techniques take advantage of some properties that are unique to DNA sequences in order to improve the compression rate, and usually perform better than general-purpose compressors. By exploring the performance of available algorithms, we produce a powerful compression tool for NGS data called MZPAQ. Results show that MZPAQ outperforms state-of-the-art tools on all benchmark datasets obtained from a recent survey in terms of compression ratio. MZPAQ offers the best compression ratios regardless of the sequencing platform or the size of the data. Conclusions Currently, MZPAQ’s strength is its higher compression ratio as well as its compatibility with all major sequencing platforms. MZPAQ is more suitable when the size of compressed data is crucial, such as long-term storage and data transfer. More efforts will be made in the future to target other aspects such as compression speed and memory utilization.

show abstract

“…In this case, quality scores usually account for more than half of the file [19], and it is no doubt noisier than the DNA sequence alone. The complexity of FASTQ [20] makes it harder to analyze or process; this will also be more thoroughly discussed in Section 3.2.…”

Section: Dna Data Compositionmentioning

confidence: 99%

BAQALC: Blockchain Applied Lossless Efficient Transmission of DNA Sequencing Data for Next Generation Medical Informatics

Lee¹,

Cho

Ikeno

et al. 2018

Applied Sciences

View full text Add to dashboard Cite

Due to the development of high-throughput DNA sequencing technology, genome-sequencing costs have been significantly reduced, which has led to a number of revolutionary advances in the genetics industry. However, the problem is that compared to the decrease in time and cost needed for DNA sequencing, the management of such large volumes of data is still an issue. Therefore, this research proposes Blockchain Applied FASTQ and FASTA Lossless Compression (BAQALC), a lossless compression algorithm that allows for the efficient transmission and storage of the immense amounts of DNA sequence data that are being generated by Next Generation Sequencing (NGS). Also, security and reliability issues exist in public sequence databases. For methods, compression ratio comparisons were determined for genetic biomarkers corresponding to the five diseases with the highest mortality rates according to the World Health Organization. The results showed an average compression ratio of approximately 12 for all the genetic datasets used. BAQALC performed especially well for lung cancer genetic markers, with a compression ratio of 17.02. BAQALC performed not only comparatively higher than widely used compression algorithms, but also higher than algorithms described in previously published research. The proposed solution is envisioned to contribute to providing an efficient and secure transmission and storage platform for next-generation medical informatics based on smart devices for both researchers and healthcare users.

show abstract

Performance comparison of sequential and parallel compression applications for DNA raw data

Cited by 9 publications

References 47 publications

Tackling the Challenges of FASTQ Referential Compression

Tackling the Challenges of FASTQ Referential Compression

MZPAQ: a FASTQ data compression tool

BAQALC: Blockchain Applied Lossless Efficient Transmission of DNA Sequencing Data for Next Generation Medical Informatics

Contact Info

Product

Resources

About