Compiling PCRE to FPGA for accelerating SNORT IDS

Abstract

Deep Payload Inspection systems like SNORT and BRO utilize regular expression for their rules due to their high expressibility and compactness. The SNORT IDS system uses the PCRE Engine for regular expression matching on the payload. The software based PCRE Engine utilizes an NFA engine based on certain opcodes which are determined by the regular expression operators in a rule. Each rule in the SNORT ruleset is translated by PCRE compiler into an unique regular expression engine. Since the software based PCRE engine can match the payload with a single regular expression at a time, and needs to do so for multiple rules in the ruleset, the throughput of the SNORT IDS system dwindles as each packet is processed through a multitude of regular expressions.In this paper we detail our implementation of hardware based regular expression engines for the SNORT IDS by transforming the PCRE opcodes generated by the PCRE compiler from SNORT regular expression rules. Our compiler generates VHDL code corresponding to the opcodes generated for the SNORT regular expression rules. We have tuned our hardware implementation to utilize an NFA based regular expression engine, using greedy quantifiers, in much the same way as the software based PCRE engine. Our system implements a regular expression only once for each new rule in the SNORT ruleset, thus resulting in a fast system that scales well with new updates. We implement two hundred PCRE engines based on a plethora of SNORT IDS rules, and use a Virtex-4 LX200 FPGA, on the SGI RASC RC 100 Blade connected to the SGI ALTIX 4700 supercomputing system as a testbed. We obtain an interface throughput of (12.9 GBits/s) and also a maximum speedup of 353X over software based PCRE execution.

A Highly Compressible Regular Expression Matching Circuit for Network Intrusion Detection Systems: An ECD-NFA approach

“…Figure 7 shows the various results for the clock rates reported against each design. Our ECD-NFA1RE design"s clock rate is about 22% higher than the next highest reported clock rate by [28], and about 78% higher than the least reported clock rate by [24] as seen in Figure 7. The throughput of the ECD-NFA design is also about 22% better than the next highest throughput reported by [28] and as seen in Figure 8.…”

“…The design in [23] memorizes the path that the trigger signal emanates from, based on specific constraints suitable for both exact string matching and complex regexp matching. A Perl Compatible Regular Expression (PCRE) compiler that converts regexps from the Snort ruleset into PCRE opcodes was implemented in [24]. The opcodes are instructions for the software based PCRE engine defined in a file called pcre_internal.h which is part of the PCRE Package.…”

“…The compiler translated the PCRE opcodes into VHSIC Hardware Description Language (VHDL) codes necessary for parallel implementation in the FPGA. Mitra et al [24] in their design used a wider input bus through an SRAM interface, which helped to increase the overall matching throughput. An automatic architectural optimisation approach was implemented by Yang et al [7] which spatially stack regexps matching circuits called REMs to form multiple character matching (MCMs) circuits.…”

A Lightweight Software Model for Signature-Based Application-Level Traffic Classification System

“…To update them, the chip must be redesigned, which usually requires high production cost. Mitra et al [14], [17] implemented an NFA-based regular-expression engine on an SGI Altix 4700 workstation with FPGA support. The throughput of the NFA improved noticeably as a result of their study, whereas the compact memory requirement was maintained.…”

High performance Deep Packet Inspection on multi-core platform

“…Intel [12] [14], to improve the performance of DPI applications. In general, none of them can be proved to have good scalability with multi-cores.…”