Dynamic compression schemes for graph coloring

Mustafa, Harun; Schilken, Ingo; Karasikov, Mikhail; Eickhoff, Carsten; Rätsch, Gunnar; Kahles, André

doi:10.1093/bioinformatics/bty632

Cited by 24 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been a couple methods that focus on constructing the color matrix in a manner that is both compressed and/or dynamic. This includes the method of Mustafa et al (2018) and Multi-BRWT (Karasikov et al, 2019). The main contribution of these methods is that they build the color matrix in a manner that is compressed and dynamic.…”

Section: Related Workmentioning

confidence: 99%

Building Large Updatable Colored de Bruijn Graphs via Merging

Muggli

Alipanahi

Boucher

2017

Preprint

View full text Add to dashboard Cite

Motivation: There exists several massive genomic and metagenomic data collection efforts, including GenomeTrakr and MetaSub, which are routinely updated with new data. To analyze such datasets, memory-efficient methods to construct and store the colored de Bruijn graph have been developed. Yet, a problem that has not been considered is constructing the colored de Bruijn graph in a scalable manner that allows new data to be added without reconstruction. This problem is important for large public datasets as scalability is needed but also the ability to update the construction is also needed. Results: We create a method for constructing and updating the colored de Bruijn graph on a very-large dataset through partitioning the data into smaller subsets, building the colored de Bruijn graph using a FM-index based representation, and succinctly merging these representations to build a single graph. The last step, merging succinctly, is the algorithmic challenge which we solve in this paper. We refer to the resulting method as VariMerge. We validate our approach, and show it produces a three-fold reduction in working space when constructing a colored de Bruijn graph for 8,000 strains. Lastly, we compare VariMerge to other competing methods -including Vari (Muggli et al., 2017), Rainbowfish (Almodaresi et al., 2017), Mantis (Pandey et al., 2018), Bloom Filter Trie (Holley et al., 2016), the method by Almodaresi et al. (2019)and Multi-BRWT (Karasikov et al., 2019) -and illustrate that VariMerge is the only method that is capable of building the colored de Bruijn graph for 16,000 strains in a manner that allows additional samples to be added. Competing methods either did not scale to this large of a dataset or cannot allow for additions without reconstruction. Availability: Our software is available under GPLv3 at https://github.com/cosmo-team/cosmo/ tree/VARI-merge.

show abstract

Section: Related Workmentioning

confidence: 99%

Building Large Updatable Colored de Bruijn Graphs via Merging

Muggli

Alipanahi

Boucher

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Considering the potential applications of colored dBGs, Boucher et al [4] proposed a succinct version of the data structure of Iqbal et al In their index, called VARI, the topology of the graph is encoded using BOSS [5], and the colors are stored separately from the dBG in a binary matrix C, in which the rows represent the kmers and the columns represent the colors. Since the work of Boucher et al, some authors have tried to compress and manipulate C even further; including that of [2], [25], [13], while others, such as [21] and [22] have proposed methods to store compressed and dynamic versions C.…”

Section: Introductionmentioning

confidence: 99%

An Index for Sequencing Reads Based on the Colored de Bruijn Graph

Díaz-Domínguez

2019

String Processing and Information Retrieval

View full text Add to dashboard Cite

In this article, we show how to transform a colored de Bruijn graph (dBG) into a practical index for processing massive sets of sequencing reads. Similar to previous works, we encode an instance of a colored dBG of the set using BOSS and a color matrix C. To reduce the space requirements, we devise an algorithm that produces a smaller and more sparse version of C. The novelties in this algorithm are (i) an incomplete coloring of the graph and (ii) a greedy coloring approach that tries to reuse the same colors for different strings when possible. We also propose two algorithms that work on top of the index; one is for reconstructing reads, and the other is for contig assembly. Experimental results show that our data structure uses about half the space of the plain representation of the set (1 Byte per DNA symbol) and that more than 99% of the reads can be reconstructed just from the index.

show abstract

“…The second category of approaches aggregate experiment information at the level of the k-mers. In such an approach, each query k-mer is independently looked up in an index to retrieve information about which experiments contain the k-mer (Holley et al, 2015;Muggli et al, 2017;Almodaresi et al, 2017;Mustafa et al, 2018;Pandey et al, 2018;Yu et al, 2018;Bradley et al, 2017;Almodaresi et al, 2018). In this context, experiments are referred to as colors, and such a data structure is sometimes called a colored de Bruijn graph.…”

Section: Introductionmentioning

confidence: 99%

Improved representation of sequence Bloom trees

Harris

Medvedev

2018

Preprint

View full text Add to dashboard Cite

Algorithmic solutions to index and search biological databases are a fundamental part of bioinformatics, providing underlying components to many end-user tools. Inexpensive next generation sequencing has filled publicly available databases such as the Sequence Read Archive beyond the capacity of traditional indexing methods. Recently, the Sequence Bloom Tree (SBT) and its derivatives were proposed as a way to efficiently index such data for queries about transcript presence. We build on the SBT framework to construct the HowDe-SBT data structure, which uses a novel partitioning of information to reduce the construction and query time as well as the size of the index. We evaluate HowDe-SBT by both proving theoretical bounds on its performance and using real RNA-seq data. Compared to previous SBT methods, HowDe-SBT can construct the index in less than 36% the time, and with 39% less space, and can answer small-batch queries at least five times faster. HowDe-SBT is available as a free open source program on https://github.com/medvedevgroup/HowDeSBT.

show abstract

Dynamic compression schemes for graph coloring

Abstract: We provide prototype implementations in C ++, summaries of our experiments as well as links to all datasets publicly at https://github.com/ratschlab/graph_annotation.

Cited by 24 publications

References 26 publications

Building Large Updatable Colored de Bruijn Graphs via Merging

Building Large Updatable Colored de Bruijn Graphs via Merging

An Index for Sequencing Reads Based on the Colored de Bruijn Graph

Improved representation of sequence Bloom trees

Contact Info

Product

Resources

About