Distributed-Memory k-mer Counting on GPUs

Abstract

A fundamental step in many bioinformatics computations is to count the frequency of fixed-length sequences, called k-mers, a problem that has received considerable attention as an important target for shared memory parallelization. With datasets growing at an exponential rate, distributed memory parallelization is becoming increasingly critical. Existing distributed memory k-mer counters do not take advantage of GPUs for accelerating computations. Additionally, they do not employ domain-specific optimizations to reduce communication volume in a distributed environment. In this paper, we present the first GPU-accelerated distributed-memory parallel k-mer counter. We evaluate the communication volume as the major bottleneck in scaling k-mer counting to multiple GPU-equipped compute nodes and implement a supermer-based optimization to reduce the communication volume and to enhance scalability. Our empirical analysis examines the balance of communication to computation on a state-of-the-art system, the Summit supercomputer at Oak Ridge National Lab. Results show overall speedups of up to two orders of magnitude with GPU optimization over CPU-based kmer counters. Furthermore, we show an additional 1.5× speedup using the supermer-based communication optimization.

Singleton Sieving: Overcoming the Memory/Speed Trade-Off in Exascale κ-mer Analysis

“…Host-Device communication is automatically handled by MetaHipMer, as k-mers are batched into compressed variants called supermers [21]. These supermers allow for efficient communication of k-mers across the network, and one kernel call is applied per supermer to unpack and insert the stored k-mers into the device hash table.…”

“…This hashing helps achieve good load balance across processes. A special minimizer-based scheme is used to reduce the communication volume [21]. Using GPUs boosts performance, but further constrains memory (e.g.…”

“…HPDA applications can greatly benefit from using filters on the GPU. For example, k-mer analysis is the very first data processing step in MetaHipMer and numerous other pipelines in computational biology [4,21]. In k-mer analysis, we start by parsing the raw sequencing data into length-k subsequences (called k-mers) and counting the occurrences of each k-mer using a GPU-based hash table.…”

“…In k-mer analysis, we start by parsing the raw sequencing data into length-k subsequences (called k-mers) and counting the occurrences of each k-mer using a GPU-based hash table. K -mer counting is used to weed out erroneous data (singleton k-mers) caused by sequencing errors, estimate sequencing depth, prepare sequencing data for assembly, and many other downstream tasks [21]. A filter is often used in k-mer counting to separate out singleton k-mers [18,20] before inserting them in the hash table.…”

Fast and accurate mapping of long reads to complete genome assemblies with VerityMap

“…Indeed, a naive indexing algorithm with the running time O(| Genome |* k ) becomes prohibitively slow in the case of accurate HiFi reads because mapping these reads is based on large k -mer sizes (e.g., k = 300). Jellyfish ( Marçais and Kingsford 2011 ), KMC3 ( Kokot et al 2017 ), and more scalable GPU–based k -mer counting approaches ( Nisa et al 2021 ) generate a database of counts that allows a constant-time count query for any k -mer. However, even though one can rapidly generate a counting database, existing implementations for indexing all rare k -mers still require O(| Genome |* k ) time.…”