Nucleotide Archival Format (NAF) enables efficient lossless reference-free compression of DNA sequences

Kryukov, Kirill; Ueda, Mahoko Takahashi; Nakagawa, So; Imanishi, Tadashi

doi:10.1101/501130

Cited by 7 publications

(6 citation statements)

References 8 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tested all DNA sequence compressors that are available and functional in 2020: dnaX [ 14 ], XM [ 15 ], DELIMINATE [ 16 ], Pufferfish [ 17 ], DNA-COMPACT [ 18 ], MFCompress [ 19 ], UHT [ 20 ], GeCo [ 21 ], GeCo2 [ 22 ], JARVIS [ 23 ], NAF [ 24 ], and NUHT [ 25 ]. We also included the relatively compact among homology search database formats: BLAST [ 26 ] and 2bit—a database format of BLAT [ 27 ].…”

Section: Resultsmentioning

confidence: 99%

Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

et al. 2020

Self Cite

View full text Add to dashboard Cite

Abstract Background Nearly all molecular sequence databases currently use gzip for data compression. Ongoing rapid accumulation of stored data calls for a more efficient compression tool. Although numerous compressors exist, both specialized and general-purpose, choosing one of them was difficult because no comprehensive analysis of their comparative advantages for sequence compression was available. Findings We systematically benchmarked 430 settings of 48 compressors (including 29 specialized sequence compressors and 19 general-purpose compressors) on representative FASTA-formatted datasets of DNA, RNA, and protein sequences. Each compressor was evaluated on 17 performance measures, including compression strength, as well as time and memory required for compression and decompression. We used 27 test datasets including individual genomes of various sizes, DNA and RNA datasets, and standard protein datasets. We summarized the results as the Sequence Compression Benchmark database (SCB database, http://kirr.dyndns.org/sequence-compression-benchmark/), which allows custom visualizations to be built for selected subsets of benchmark results. Conclusion We found that modern compressors offer a large improvement in compactness and speed compared to gzip. Our benchmark allows compressors and their settings to be compared using a variety of performance measures, offering the opportunity to select the optimal compressor on the basis of the data type and usage scenario specific to a particular application.

show abstract

Section: Resultsmentioning

confidence: 99%

Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…NAF is a tool and a format for storing compressed DNA sequences. 8 The tool supports multiple formats for compression. The algorithm works in lossless reference-free mode.…”

Section: Literature Reviewmentioning

confidence: 99%

SCA-NGS: Secure compression algorithm for next generation sequencing data using genetic operators and block sorting

Sardaraz

Tahir

2021

Science Progress

View full text Add to dashboard Cite

Recent advancements in sequencing methods have led to significant increase in sequencing data. Increase in sequencing data leads to research challenges such as storage, transfer, processing, etc. data compression techniques have been opted to cope with the storage of these data. There have been good achievements in compression ratio and execution time. This fast-paced advancement has raised major concerns about the security of data. Confidentiality, integrity, authenticity of data needs to be ensured. This paper presents a novel lossless reference-free algorithm that focuses on data compression along with encryption to achieve security in addition to other parameters. The proposed algorithm uses preprocessing of data before applying general-purpose compression library. Genetic algorithm is used to encrypt the data. The technique is validated with experimental results on benchmark datasets. Comparative analysis with state-of-the-art techniques is presented. The results show that the proposed method achieves better results in comparison to existing methods.

show abstract

“…CoMSA is another compression algorithm for FASTA files introduced by Deorowicz et al (2018) based on a generalized Burrows-Wheeler transform. Similarly to MFCompress, The Nucleotide Archival Format (NAF) introduced by Kryukov et al (2019) is another compressor that works on amino acid sequences converted to their corresponding DNA bases by dictionary encoding this transformed string.…”

Section: Introductionmentioning

confidence: 99%

Image-centric compression of protein structures improves space savings

Staniscia

2022

Preprint

View full text Add to dashboard Cite

Motivation: Because of the rapid generation of data, the study of compression algorithms to reduce storage and transmission costs is important to bioinformaticians. Much of the focus has been on sequence data, including both genomes and protein amino acid sequences stored in FASTA files. However, there are few specialized compressors for structural protein data contained in PDB and mmCIF files. Current standard practice is to use an ordinary lossless compressors such as gZip on a sequential list of atomic coordinates, but this approach expends bits on saving an arbitrary ordering of atoms, and it also prevents reordering the atoms for compressibility. In this article, we demonstrate for the first time that image-based compression can significantly improve compression ratios. To this end, we implement a prototype compressor 'PIC', specialized for point clouds of atom coordinates contained in PDB and mmCIF files. PIC maps the 3D data to a 2D 8-bit greyscale image and leverages the well developed PNG image compressor to minimize the size of the resulting image, forming the compressed file. Results: PIC outperforms gZip in terms of compression ratio on proteins over 20,000 atoms in size, with a savings over gZip of up to 37.4% on the proteins compressed. In addition, PIC's compression ratio increases with protein size.

show abstract

Nucleotide Archival Format (NAF) enables efficient lossless reference-free compression of DNA sequences

Cited by 7 publications

References 8 publications

Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

SCA-NGS: Secure compression algorithm for next generation sequencing data using genetic operators and block sorting

Image-centric compression of protein structures improves space savings

Contact Info

Product

Resources

About