Clustering-Based Compression for Population DNA Sequences

Cheng, Kin-On; Law, Ngai-Fong; Siu, Wan-Chi

doi:10.1109/tcbb.2017.2762302

Cited by 14 publications

(15 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also attempted to test the performance of RCC [17] on the same genome databases. However, it was not runnable for the compression of long genome sequences (such as human and rice) due to its time complexity— RCC was taking longer than 10 h to compress only four human genome sequences.…”

Section: Resultsmentioning

confidence: 99%

“…The third category of algorithms selects reference via unsupervised learning. RCC [17] performs clustering on the local histogram of dataset and derives a representative sequence of each cluster as the reference sequence for the corresponding cluster. A final representative sequence is then selected from the representative sequence set.…”

Section: Related Workmentioning

confidence: 99%

“…Reference-based genome compression for compressing a single genome sequence has been intensively studied and achieved much higher compression ratio than reference free compression [8]. Existing reference-based genome compression algorithms include GDC [9], GDC2 [10], iDoComp [11], ERGC [12], HiRGC [13], CoGI [14], RlZAP [15], MSC [16], RCC [17], NRGC [18], SCCG [19] and FRESCO [20]. A straightforward application of these reference-based compression algorithms to solve the challenging problem of compressing a database containing n number of genome sequences is to conduct a one-by-one sequential reference-based compression for every genome in the database using one fixed reference genome.…”

Section: Introductionmentioning

confidence: 99%

“…The key differences between ECC and other compression schemes for sequence databases such as MSC [16] and RCC [17] include: (i) Our estimation on pairwise sequence distances is based on the sketch distance of the reduced k -mer sets [21] instead of the Euclidean distance between vectors of k -mer frequencies [17]; (ii) Our initial setting of the centroid in the clustering is not randomly as by RCC, but determined by the analysis on the whole database;(iii) The reference selection within the clusters is also decided by the clustering method instead of the reconstruction of the original target genome set by RCC.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Sketch distance-based clustering of chromosomes for large genome database compression

et al. 2019

View full text Add to dashboard Cite

Background: The rapid development of Next-Generation Sequencing technologies enables sequencing genomes with low cost. The dramatically increasing amount of sequencing data raised crucial needs for efficient compression algorithms. Reference-based compression algorithms have exhibited outstanding performance on compressing single genomes. However, for the more challenging and more useful problem of compressing a large collection of n genomes, straightforward application of these reference-based algorithms suffers a series of issues such as difficult reference selection and remarkable performance variation. Results: We propose an efficient clustering-based reference selection algorithm for reference-based compression within separate clusters of the n genomes. This method clusters the genomes into subsets of highly similar genomes using MinHash sketch distance, and uses the centroid sequence of each cluster as the reference genome for an outstanding reference-based compression of the remaining genomes in each cluster. A final reference is then selected from these reference genomes for the compression of the remaining reference genomes. Our method significantly improved the performance of the-state-of-art compression algorithms on large-scale human and rice genome databases containing thousands of genome sequences. The compression ratio gain can reach up to 20-30% in most cases for the datasets from NCBI, the 1000 Human Genomes Project and the 3000 Rice Genomes Project. The best improvement boosts the performance from 351.74 compression folds to 443.51 folds. Conclusions: The compression ratio of reference-based compression on large scale genome datasets can be improved via reference selection by applying appropriate data preprocessing and clustering methods. Our algorithm provides an efficient way to compress large genome database.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sketch distance-based clustering of chromosomes for large genome database compression

et al. 2019

View full text Add to dashboard Cite

show abstract

“…In other words, similar subsequences are encoded once only which act as references to their occurrence at other locations of the same sequence or other sequences. Examples of methods in this group include DNAzip [12], RLCSA [46], RLZ [47,48], GRS [49], GReEn [50], iDoComp [51], COMRAD [52,53], ERGC [54], CoGI [55], MSC [56], GDC [57,58], FRESCO [59] and RCC [60]. With the use of appropriate reference sequences, the storage size reduction in some cases can be over 90%.…”

Section: Introductionmentioning

confidence: 99%

Application of signal processing for DNA sequence compression

Law

2019

IET signal process.

Self Cite

View full text Add to dashboard Cite

Due to the advancement of high-throughput sequencing technologies, it is now feasible for sequencing individual genomes in a fast and affordable manner. With the significant increase in the number of individual genomes, compression methods are needed to reduce pressure on data storage as well as enable effective data distribution and management. The compression methods can generally be divided into two classes, namely reference-free methods and reference-based methods. In reference-free methods, redundancies within the target DNA sequence to be compressed are explored. In reference-based methods, redundancies between the target DNA sequence and other reference sequences are identified to achieve compression. This type of method is applicable to population sequences which are highly similar to each other and have a small number of mismatches. Some of the methods can also be applied to partially similar sequences such as chromosome sequences or sequences having evolutionary relationship. The authors highlight recent developments in these methods. In the comparative study, the authors' simulation results reveal that the selection of a reference sequence is a crucial factor affecting the compression performance. Use of multiple number of reference sequences and enhancement strategies such as reference rewriting are important to achieve a large compression gain. (a) Short segment of the chromosome 19 of the human Homo sapiens genome, (b) Its lossless encoded representation with 'A' represented as '00', 'T' as '11', 'C' as '01' and 'G' as '10', (c) Resultant segment after the first referencing instruction is applied

show abstract