GaKCo: A Fast Gapped k-mer String Kernel Using Counting

Abstract

Abstract. String Kernel (SK) techniques, especially those using gapped k-mers as features (gk), have obtained great success in classifying sequences like DNA, protein, and text. However, the state-of-the-art gk-SK runs extremely slow when we increase the dictionary size (Σ) or allow more mismatches (M ). This is because current gk-SK uses a trie-based algorithm to calculate cooccurrence of mismatched substrings resulting in a time cost proportional to O(Σ M ). We propose a fast algorithm for calculating Gapped k-mer Kernel using Counting (GaKCo). GaKCo uses associative arrays to calculate the co-occurrence of substrings using cumulative counting. This algorithm is fast, scalable to larger Σ and M , and naturally parallelizable. We provide a rigorous asymptotic analysis that compares GaKCo with the state-of-the-art gk-SK. Theoretically, the time cost of GaKCo is independent of the Σ M term that slows down the trie-based approach. Experimentally, we observe that GaKCo achieves the same accuracy as the state-of-the-art and outperforms its speed by factors of 2, 100, and 4, on classifying sequences of DNA (5 datasets), protein (12 datasets), and character-based English text (2 datasets).

FastSK: Fast Sequence Analysis with Gapped String Kernels

“…BaselinesWe compare the prediction accuracy and efficiency of FastSK with 3 state-of-the-art string kernel baselines. For DNA and protein data, we baseline against gkmSVM-2.0 [8] and GaKCo [27]. For an NLP string kernel baseline, we use the Blended Spectrum Kernel [12,11], as it has recently achieved strong results in natural language processing.…”

“…FastSK directly counts the gapped k-mers shared between sequences, previous works (e.g. [7,8,18,19,27]) indirectly compute the kernel function by inferring the counts from a set of mismatch statistics. These methods take inspiration from [17], which uses the notion of a mismatch neighborhood to efficiently compute the (k, m)-mismatch kernel.…”

“…Counting ImplementationsGaKCo [27] is similar to FastSK in that it uses uses a sort-and-count algorithm. However, it differs from FastSK in that it follows the mismatch statistic formulation from [7].…”

“…String kernels in conjunction with Support Vector Machines (SK-SVM) achieve strong prediction performance across a variety of sequence analysis tasks, with widespread use in bioinformatics and natural language processing (NLP). SK-SVMs are a popular technique for DNA regulatory element identification [22,10,25,7,19,27,8], and bio-medical named entity recognition [23,15]. SK-SVMs are also popular baselines for evaluating the quality of deep learning models [3,10,25] for analyzing variant impacts.…”

“…Therefore, the feature vectors are both extremely large and extremely sparse, which makes models trained on these feature vectors highly prone to overfitting and poor generalization. Third, existing algorithms that overcome these challenges leave much to be desired; for example, popular trie-based approaches (e.g., [7,4]) still exhibit exponential dependence on |Σ| [20,27], k, and m. On the other hand, counting-based methods rely on complex "mismatch statistics" to indirectly obtain feature counts [17,6,27], however still fail to scale to greater feature lengths. Together, these issues present major limitations to the practical utility of k-mer string kernel methods.…”