PHONI: Streamed Matching Statistics with Multi-Genome References

Boucher, Christina; Gagie, Travis; Tomohiro, I; Köppl, Dominik; Langmead, Ben; Manzini, Giovanni; Navarro, Gonzalo; Pacheco, Alejandro; Rossi, Massimiliano

doi:10.1109/dcc50243.2021.00027

Cited by 16 publications

(31 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Now we show how we can compute eMS extending the algorithm presented in Boucher et al [3] while preserving the same space-bound.…”

Section: Computing the Second Longest Matchmentioning

confidence: 92%

“…From now on, we refer to the set of all maximal unique matches between T and P as MUMs. In [3] the authors showed how to compute maximal matches (not necessarily unique neither in T nor P ) in O(r + g) space, where r is the number of runs of the BWT of T and g is the size of the SLP representing the text T . This is achieved by computing the matching statistics, for which we report the definition given in [3].…”

Section: Definitionmentioning

confidence: 99%

“…We can apply verbatim the algorithm of [3] to compute the eMS[i].pos and eMS [i].len while we extend the algorithm to include the computation of eMS[i].slen. The following Lemma shows how to find the second longest match using the LCP array.…”

Section: Computing the Second Longest Matchmentioning

confidence: 99%

See 2 more Smart Citations

Computing Maximal Unique Matches with the r-index

Giuliani¹,

Romana²,

Rossi³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, pangenomes received increasing attention from the scientific community for their ability to incorporate population variation information and alleviate reference genome bias. Maximal Exact Matches (MEMs) and Maximal Unique Matches (MUMs) have proven themselves to be useful in multiple bioinformatic contexts, for example short-read alignment and multiple-genome alignment. However, standard techniques using suffix trees and FM-indexes do not scale to a pangenomic level. Recently, Gagie et al. [JACM 20] introduced the r-index that is a Burrows-Wheeler Transform (BWT)-based index able to handle hundreds of human genomes. Later, Rossi et al. [JCB 22] enabled the computation of MEMs using the r-index, and Boucher et al. [DCC 21] showed how to compute them in a streaming fashion.In this paper, we show how to augment Boucher et al.'s approach to enable the computation of MUMs on the r-index, while preserving the space and time bounds. We add additional O(r) samples of the longest common prefix (LCP) array, where r is the number of equal-letter runs of the BWT, that permits the computation of the second longest match of the pattern suffix with respect to the input text, which in turn allows the computation of candidate MUMs. We implemented a proof-of-concept of our approach, that we call mum-phinder, and tested on real-world datasets. We compared our approach with competing methods that are able to compute MUMs. We observe that our method is up to 8 times smaller, while up to 19 times slower when the dataset is not highly repetitive, while on highly repetitive data, our method is up to 6.5 times slower and uses up to 25 times less memory. ACM Subject Classification Theory of computation → Data structures design and analysisKeywords and phrases Burrows-Wheeler Transform, r-index, maximal unique matches, bioinformatics, pangenomics Supplementary Material Software: https://github.com/saragiuliani/mum-phinder

show abstract

“…Now we show how we can compute eMS extending the algorithm presented in Boucher et al [3] while preserving the same space-bound.…”

Section: Computing the Second Longest Matchmentioning

confidence: 92%

Section: Definitionmentioning

confidence: 99%

See 1 more Smart Citation

Computing Maximal Unique Matches with the r-index

Giuliani¹,

Romana²,

Rossi³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…They used a balanced straight-line program (SLP) for T to support random access in O(log n) time, so MONI finds all MEMs of P with respect to T in O(m log n) time. In a separate paper [4], they and their coauthors observed that if the SLP is used to support longest longest-common-extension (LCE) queries in O(log 2 n) time, then MONI needs only one pass over P and O(m log 2 n) time. They named one-pass implementation PHONI (for "PHony MONI", and because they considered running it on smartPHOnes) because of the increased running time, but later realized that if the SLP is locally consistent as well as balanced then the LCE queries take O(log n) time.…”

Section: Introductionmentioning

confidence: 99%

MONI can find k-MEMs

Gagie¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Maximal exact matches (MEMs) have been widely used in bioinformatics at least since Li (2013) presented BWA-MEM. Building on work by Bannai, Gagie and I (2018), recently built an index called MONI, based on the run-length compressed Burrows-Wheeler Transform, that can find MEMs efficiently with respect to pangenomes. In this paper we define k-MEMs to be maximal substrings of a pattern that each occur exactly at k times in a text (so a MEM is a 1-MEM) and show that, when k is given at construction time, MONI can find k-MEMs efficiently as well.

show abstract

“…The increase in the amount of highly compressible data that requires efficient processing in the recent years, particularly in the area of computational genomics [3,4], has caused a spike of interest in dictionary compression. Its main idea is to reduce the size of the representation of data by finding repetitions in the input and encoding them as references to other occurrences.…”

Section: Introductionmentioning

confidence: 99%

Fast and Space-Efficient Construction of AVL Grammars from the LZ77 Parsing

Kempa,

Langmead

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Grammar compression is, next to Lempel-Ziv (LZ77) and run-length Burrows-Wheeler transform (RLBWT), one of the most flexible approaches to representing and processing highly compressible strings. The main idea is to represent a text as a context-free grammar whose language is precisely the input string. This is called a straight-line grammar (SLG). An AVL grammar, proposed by Rytter [Theor. Comput. Sci., 2003] is a type of SLG that additionally satisfies the AVL-property: the heights of parse-trees for children of every nonterminal differ by at most one. In contrast to other SLG constructions, AVL grammars can be constructed from the LZ77 parsing in compressed time: O(z log n) where z is the size of the LZ77 parsing and n is the length of the input text. Despite these advantages, AVL grammars are thought to be too large to be practical.We present a new technique for rapidly constructing a small AVL grammar from an LZ77 or LZ77-like parse. Our algorithm produces grammars that are always at least five times smaller than those produced by the original algorithm, and never more than double the size of grammars produced by the practical Re-Pair compressor [Larsson and Moffat, Proc. IEEE, 2000]. Our algorithm also achieves low peak RAM usage. By combining this algorithm with recent advances in approximating the LZ77 parsing, we show that our method has the potential to construct a run-length BWT from an LZ77 parse in about one third of the time and peak RAM required by other approaches. Overall, we show that AVL grammars are surprisingly practical, opening the door to much faster construction of key compressed data structures.

show abstract

PHONI: Streamed Matching Statistics with Multi-Genome References

Cited by 16 publications

References 17 publications

Computing Maximal Unique Matches with the r-index

Computing Maximal Unique Matches with the r-index

MONI can find k-MEMs

Fast and Space-Efficient Construction of AVL Grammars from the LZ77 Parsing

Contact Info

Product

Resources

About