Regular Expression Matching using Bit Vector Automata

Abstract

Regular expressions (regexes) are ubiquitous in modern software. There is a variety of implementation techniques for regex matching, which can be roughly categorized as (1) relying on backtracking search, or (2) being based on finite-state automata. The implementations that use backtracking are often chosen due to their ability to support advanced pattern-matching constructs. Unfortunately, they are known to suffer from severe performance problems. For some regular expressions, the running time for matching can be exponential in the size of the input text. In order to provide stronger guarantees of matching efficiency, automata-based regex matching is the preferred choice. However, even these regex engines may exhibit severe performance degradation for some patterns. The main reason for this is that regexes used in practice are not exclusively built from the classical regular constructs, i.e., concatenation, nondeterministic choice and Kleene's star. They involve additional constructs that provide succinctness and convenience of expression. The most common such construct is bounded repetition (also called counting), which describes the repetition of the pattern a fixed number of times. In this paper, we propose a new algorithm for the efficient matching of regular expressions that involve bounded repetition. Our algorithms are based on a new model of automata, which we call nondeterministic bit vector automata (NBVA). This model is chosen to be expressively equivalent to nondeterministic counter automata with bounded counters, a very natural model for expressing patterns with bounded repetition. We show that there is a class of regular expressions with bounded repetition that can be matched in time that is independent from the repetition bounds. Our algorithms are general enough to cover the vast majority of challenging bounded repetitions that arise in practice. We provide an implementation of our approach in a regex engine, which we call BVA-Scan. We compare BVA-Scan against state-of-the-art regex engines on several real datasets.

One Automaton to Rule Them All: Beyond Multiple Regular Expressions Execution

“…To fully exploit FSAs executive potential, there exist various techniques fostering the optimization of both NFAs and DFAs that tackle NFAs partitioning [26], multi-stride DFAs [11], [28], [40], and DFAs compression [33], [46]. These optimization approaches involve both architectural [6], [13], [20]- [23], [26], [41], [47] and algorithmic aspects [11], [12], [14], [33]. An architectural approach to optimizing NFAs [26] relies on a toolchain to partition them, depending on the available hardware resources and units.…”

“…Differently, Wang et al [6] approach optimize pattern matching via REs decomposition to extract simpler strings out of complex ones, matching them separately and eventually re-composing their results. Others focus on the execution improvements of counting in REs and FSAs [12]- [14] focusing either on decomposing at the bit level the automaton or on software-hardware codesigned in-memory architectures.…”

“…However, to do so, REs rely on complex operators, e.g., describing (un)bounded repetitions of characters and sub-REs. These characteristics make pattern matching more complex than basic string matching with its simple character concatenation [12]- [14], which is instead a welldefined problem addressed by various existing algorithms [15]. In this context, the parallelization [6], [16]- [23] of pattern matching represents a valuable opportunity, increasing the Fig.…”