An improved algorithm to accelerate regular expression evaluation

Becchi, Michela; Crowley, Patrick

doi:10.1145/1323548.1323573

Cited by 182 publications

(164 citation statements)

References 17 publications

Supporting

Mentioning

152

Contrasting

Unclassified

Order By: Relevance

“…However, state-space explosion of the resulting DFA can greatly increase the amount of required memory, which could in turn impact the memory performance and even the practical feasibility of the solution. Various techniques were proposed to compress the state transition table (STT) of the DFA with nondeterministic features [7,10,12,13,22]. These techniques, however, can be computationally expensive and are often designed a-posteriori to a particular set (or type) of regexes.…”

Section: Related Workmentioning

confidence: 99%

“…Several techniques [7,13,22] have been proposed to compress the STT by introducing nondeterminism to the DFA. These nondeterministic features usually require more complex run-time processing and multiple random memory accesses per input character; both can reduce matching throughput.…”

Section: Introductionmentioning

confidence: 99%

“…The compression ratio is usually highly regex-dependent, and the compressed DFA size still much larger than the original NFA. The DFA compression algorithm can be complex (e.g., quadratic in the number of DFA states [7]), making regex update even more difficult.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimizing Regular Expression Matching with SR-NFA on Multi-Core Systems

Yang

Prasanna

2011

2011 International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

Abstract-Conventionally, regular expression matching (REM) has been performed by sequentially comparing the regular expression (regex) to the input stream, which can be slow due to excessive backtracking [21]. Alternatively, the regex can be converted to a deterministic finite automaton (DFA) for efficient matching, which however may require an extremely large state transition table (STT) due to exponential state explosion [17,27]. We propose the segmented regex-NFA (SR-NFA) architecture, where the regex is first compiled into modular nondeterministic finite automata (NFA), then partitioned, optimized, and matched efficiently on modern multi-core processors. SR-NFA offers attack-resilient multi-gigabit per second matching throughput, does not suffer from either backtracking or state explosion, and can be rapidly constructed. For regex sets that construct a DFA with moderate state explosion, i.e., on average 200k states in the STT, the proposed SR-NFA is 367k times faster to construct and update and use 23k times less memory than the DFA approach. Running on an 8-core 2.6 GHz Opteron platform, our prototype achieves 2.2 Gbps average matching throughput for regex sets with up to 4,000 SR-NFA states per regex set.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimizing Regular Expression Matching with SR-NFA on Multi-Core Systems

Yang

Prasanna

2011

2011 International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

show abstract

“…D 2 FA [7] acknowledged as the original work in this approach, is proposed to compress DFA by applying default transition, but at a cost of accessing DFA multiple times per input character. Based on it, later works including [8], [9] improve the worst case performance.…”

Section: Related Work and Backgroundmentioning

confidence: 99%

Parallel DFA Architecture for Ultra High Throughput DFA-Based Pattern Matching

Tang

Jiang

Wang

et al. 2010

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYMulti-pattern matching is a key technique for implementing network security applications such as Network Intrusion Detection/ Protection Systems (NIDS/NIPSes) where every packet is inspected against tens of thousands of predefined attack signatures written in regular expressions (regexes). To this end, Deterministic Finite Automaton (DFA) is widely used for multi-regex matching, but existing DFA-based researches have claimed high throughput at an expense of extremely high memory cost, so fail to be employed in devices such as high-speed routers and embedded systems where the available memory is quite limited. In this paper, we propose a parallel architecture of DFA called Parallel DFA (PDFA) taking advantage of the large amount of concurrent flows to increase the throughput with nearly no extra memory cost. The basic idea is to selectively store the underlying DFA in memory modules that can be accessed in parallel. To explore its potential parallelism we intensively study DFAsplit schemes from both state and transition points in this paper. The performance of our approach in both the average cases and the worst cases is analyzed, optimized and evaluated by numerical results. The evaluation shows that we obtain an average speedup of 100 times compared with traditional DFA-based matching approach.

show abstract

“…Further more, it is bound to a precise state encoding, not a general and broadly applicable. The second improvement is proposed in [9]. They first define the depth of states, and then add the constraint that the default transitions can be oriented only in the direction of deceasing depth.…”

Section: Related Workmentioning

confidence: 99%

Fast and Memory-Efficient Regular Expression Matching Using Transition Sharing

Zhang

Luo

Fang

et al. 2009

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYScanning packet payload at a high speed has become a crucial task in modern network management due to its wide variety applications on network security and application-specific services. Traditionally, Deterministic finite automatons (DFAs) are used to perform this operation in linear time. However, the memory requirements of DFAs are prohibitively high for patterns used in practical packet scanning, especially when many patterns are compiled into a single DFA. Existing solutions for memory blow-up are making a trade-off between memory requirement and memory access of processing per input character. In this paper we proposed a novel method to drastically reduce the memory requirements of DFAs while still maintain the high matching speed and provide worst-case guarantees. We removed the duplicate transitions between states by dividing all the DFA states into a number of groups and making each group of states share a merged transition table. We also proposed an efficient algorithm for transition sharing between states. The high efficiency in time and space made our approach adapted to frequently updated DFAs. We performed several experiments on real world rule sets. Overall, for all rule sets and approach evaluated, our approach offers the best memory versus run-time trade-offs.

show abstract

An improved algorithm to accelerate regular expression evaluation

Cited by 182 publications

References 17 publications

Optimizing Regular Expression Matching with SR-NFA on Multi-Core Systems

Optimizing Regular Expression Matching with SR-NFA on Multi-Core Systems

Parallel DFA Architecture for Ultra High Throughput DFA-Based Pattern Matching

Fast and Memory-Efficient Regular Expression Matching Using Transition Sharing

Contact Info

Product

Resources

About