Why GPUs are Slow at Executing NFAs and How to Make them Faster

Abstract

Non-deterministic Finite Automata (NFA) are space-efficient finite state machines that have significant applications in domains such as pattern matching and data analytics. In this paper, we investigate why the Graphics Processing Unit (GPU)-a massively parallel computational device with the highest memory bandwidth available on generalpurpose processors-cannot efficiently execute NFAs. First, we identify excessive data movement in the GPU memory hierarchy and describe how to privatize reads effectively using GPU's on-chip memory hierarchy to reduce this excessive data movement. We also show that in several cases, indirect table lookups in NFAs can be eliminated by converting memory reads into computation, to further reduce the number of memory reads. Although our optimization techniques significantly alleviate these memory-related bottlenecks, a side effect of these techniques is the static assignment of work to cores. This leads to poor compute utilization, where GPU cores are wasted on idle NFA states. Therefore, we propose a new dynamic scheme that effectively balances compute utilization with reduced memory usage. Our combined optimizations provide a significant improvement over the previous state-ofthe-art GPU implementations of NFAs. Moreover, they enable current GPUs to outperform the domain-specific accelerator for NFAs (i.e., Automata Processor) across several applications while performing within an order of magnitude for the rest of the applications. CCS Concepts • Computing methodologies → Parallel algorithms; • Computer systems organization → Single instruction, multiple data; • Theory of computation → Formal languages and automata theory.

YARB: a Methodology to Characterize Regular Expression Matching on Heterogeneous Systems

“…Many works aim to exploit these parallelism degrees on top of a massively parallel architecture such as GPUs. We exclude closed-source approaches [17], [36], while selecting the approach proposed by Liu et al [31] which exposes several state-of-the-art GPU-based methodologies.…”

“…We select the available engines as described in §II and represented in A of Figure 1. Indeed, YARB's engines currently are: RE2 [30], Hyperscan [14], and the ones presented by Liu et al [31]. RE2 is a C ++ general-purpose library that guarantees execution time linear in the input length and fixed stack footprint, able to target any CPU regardless the ISA.…”

“…Since we adopt REs as universal language, we devise a novel Automata to REs translator that can optionally apply different minimization algorithms and explore solutions' performance. We analyze open-source literature for RE matching on heterogeneous systems and select the ones working on every scenario [14], [30], [31], providing relevant insights. In summary, this work's contributions are:…”

Software-hardware codesign for efficient in-memory regular pattern matching

“…Our notion of counter-ambiguity is formulated more generally, and our simulation based on bit vectors handles character class ambiguity. Finally, there are several works that implement regex matching algorithms on GPUs [14,29,60,70].…”

High-performance and energy-efficient irregular graph processing on GPU architectures

“…Specialized approaches instead focus on a selected application and exploit the characteristics of it. Works on Non-deterministic finite automaton (NFA) propose to dynamically employ the GPU shared memory to store frequently used sizable lookup tables [91]. Many specialized works have focused on GPU execution of irregular Sparse Matrix Vector Multiplication (SpMV) and Matrix Matrix Multiplication (GEMM) by proposing software approaches that reorder the matrices dataset [119], algorithms tailored for specific data characteristics of the matrices [127], and row reordering techniques [69] to improve data locality among processed rows.…”