EPR-Dictionaries: A Practical and Fast Data Structure for Constant Time Searches in Unidirectional and Bidirectional FM Indices

Abstract

The unidirectional FM index was introduced by Ferragina and Manzini in 2000 and allows to search a pattern in the index in one direction. The bidirectional FM index (2FM) was introduced by Lam et al. in 2009. It allows to search for a pattern by extending an infix of the pattern arbitrarily to the left or right. The method of Lam et al. can conduct one step in time O(σ) while needing space O(σ · n) using constant time rank queries on bit vectors. Schnattinger and colleagues improved this time to O(log σ) while usingO(log σ · n) bits of space for both, the FM and 2FM index. This is achieved by the use of binary wavelet trees. In this paper we introduce a new, practical method for conducting an exact search in a uni-and bidirectional FM index in O(1) time per step while using O(log σ · n) + o(log σ · σ · n) bits of space. This is done by replacing the binary wavelet tree by a new data structure, the Enhanced Prefixsum Rank dictionary (EPR-dictionary). We implemented this method in the SeqAn C++ library and experimentally validated our theoretical results. In addition we compared our implementation with other freely available implementations of bidirectional indices and show that we are between ≈ 2.6 − 4.8 times faster. This will have a large impact for many bioinformatics applications that rely on practical implementations of (2)FM indices e.g. for read mapping. To our knowledge this is the first implementation of a constant time method for a search step in 2FM indices.

Dynamic partitioning of search patterns for approximate pattern matching using search schemes

“…By keeping track of both the range over the BWT as well as the range over BWT r in a synchronized manner ( Lam et al., 2009 ), one can extend a pattern P to both as well as . By replacing the ‘Occ’ data structure with a ‘Prefix-Occ’ structure, this can be realized in constant time ( Pockrandt et al, 2017 ).…”

Pan-genome de Bruijn Graph using the Bidirectional FM-index

“…Note that all variables related to the reverse part of the bidirectional FM-index are denoted with a superscript r. Bit vectors B and B r will be explained later. Exact 1) time [40]. In other words, the key functionality of a bidirectional FM-index entails that a partial match can be extended with a character either to the left or to the right.…”

GenMap: Fast and Exact Computation of Genome Mappability

“…To allow searches that start with a middle piece as illustrated in figure 3c a bidirectional index is required again. To improve the overall running time of the index-based search we use a fast implementation of bidirectional FM indices based on EPR dictionaries [8].…”