Fast and efficient short read mapping based on a succinct hash index

Abstract

BackgroundVarious indexing techniques have been applied by next generation sequencing read mapping tools. The choice of a particular data structure is a trade-off between memory consumption, mapping throughput, and construction time.ResultsWe present the succinct hash index – a novel data structure for read mapping which is a variant of the classical q-gram index with a particularly small memory footprint occupying between 3.5 and 5.3 GB for a human reference genome for typical parameter settings. The succinct hash index features two novel seed selection algorithms (group seeding and variable-length seeding) and an efficient parallel construction algorithm, which we have implemented to design the FEM (Fast(F) and Efficient(E) read Mapper(M)) mapper. FEM can return all read mappings within a given edit distance. Our experimental results show that FEM is scalable and outperforms other state-of-the-art all-mappers in terms of both speed and memory footprint. Compared to Masai, FEM is an order-of-magnitude faster using a single thread and two orders-of-magnitude faster when using multiple threads. Furthermore, we observe an up to 2.8-fold speedup compared to BitMapper and an order-of-magnitude speedup compared to BitMapper2 and Hobbes3.ConclusionsThe presented succinct index is the first feasible implementation of the q-gram index functionality that occupies around 3.5 GB of memory for a whole human reference genome. FEM is freely available at https://github.com/haowenz/FEM.

Simulation and Modeling of Telocytes Behavior in Signaling and Intercellular Communication Processes

“…Hash indexes are efficient with precise values searching queries, but the returned values are not sorted. Hash indexes are optimized for queries that use the equality operator and they support full index scans [39].…”

srnaMapper: an optimal mapping tool for sRNA-Seq reads

“…We choose an error rate of 10 (which is two errors at most for a read or size 20), and discarded reads with more that 2 erros. Other all mappers, such as FEM (Zhang et al, 2018), Hobbes (Ahmadi et al, 2011), and BitMapper2 (Cheng et al, 2019), could not be used, because the reads were too short for an edit distance of 2.…”

Next Generation Sequencing Technology in the Clinic and Its Challenges

“…Briefly explained, FM-index alignment tools are derived from the Burrows-Wheeler Transform [ 68 ]—a method to sufficiently compress large amount of data and finding approximate matches of sequences in the reference genome [ 69 ]. Hash table-based aligners uses the seed-and-extend method in combination with additional alignment algorithms [ 68 , 70 , 71 ].…”