Regex matching with counting-set automata

Abstract

We propose a solution to the problem of efficient matching regular expressions (regexes) with bounded repetition, such as (ab){1,100}, using deterministic automata. For this, we introduce novel counting-set automata (CsAs) , automata with registers that can hold sets of bounded integers and can be manipulated by a limited portfolio of constant-time operations. We present an algorithm that compiles a large sub-class of regexes to deterministic CsAs. This includes (1) a novel Antimirov-style translation of regexes with counting to counting automata (CAs) , nondeterministic automata with bounded counters, and (2) our main technical contribution, a determinization of CAs that outputs CsAs. The main advantage of this workflow is that the size of the produced CsAs does not depend on the repetition bounds used in the regex (while the size of the DFA is exponential to them). Our experimental results confirm that deterministic CsAs produced from practical regexes with repetition are indeed vastly smaller than the corresponding DFAs. More importantly, our prototype matcher based on CsA simulation handles practical regexes with repetition regardless of sizes of counter bounds. It easily copes with regexes with repetition where state-of-the-art matchers struggle.

Software-hardware codesign for efficient in-memory regular pattern matching

“…DFAs and NFAs have been extended by [23] and [8] respectively by introducing counting operations and guards as an alternative to unfolding for large repetition bounds. An implementation of a class of counter automata, proposed in [59], is based on queues for representing sets of counter values. A variety of software regex matchers, including RE2 [18,41], Rust's Regex [44], PCRE [37], SRM [45], and Hyperscan [65] support the matching of regexes with counting.…”

Fast Matching of Regular Patterns with Synchronizing Counting (Technical Report)

“…Moreover, note that, since k is written as a decadic numeral, its value is exponential in the size of the regex. This makes matching with already moderately high k prone to significant slowdowns and ReDoS vulnerabilities with virtually every mainstream matcher (see [23,13]). At the same time, repetition bounds easily reach thousands, in extreme tens of millions (in real-life XML [24]).…”

“…The problem of matching with bounded repetition has been addressed from the theoretical as well as from the practical perspective by a number of authors [25,24,26,27,28,29,30,23]. From these, the recent work [23] is the only one offering fast matching for a practically significant class of regexes.…”

“…The problem of matching with bounded repetition has been addressed from the theoretical as well as from the practical perspective by a number of authors [25,24,26,27,28,29,30,23]. From these, the recent work [23] is the only one offering fast matching for a practically significant class of regexes. The algorithm of [23] compiles a regex with counting to a non-deterministic counting automaton (CA), an automaton with counters that can be incremented, reset, and compared with a constant.…”

“…From these, the recent work [23] is the only one offering fast matching for a practically significant class of regexes. The algorithm of [23] compiles a regex with counting to a non-deterministic counting automaton (CA), an automaton with counters that can be incremented, reset, and compared with a constant. The crux of the problem is then to convert the CA to a succinct deterministic machine that could be simulated fast in matching.…”