Efficient parsing with parser combinators

Abstract

Parser combinators offer a universal and flexible approach to parsing. They follow the structure of an underlying grammar, are modular, well-structured, easy to maintain, and can recognize a large variety of languages including context-sensitive ones. However, these advantages introduce a noticeable performance overhead mainly because the same powerful parsing algorithm is used to recognize even simple languages. Time-wise, parser combinators cannot compete with parsers generated by well-performing parser generators or optimized handwritten code. Techniques exist to achieve a linear asymptotic performance of parser combinators, yet there is a significant constant multiplier. The multiplier can be lowered to some degree, but this requires advanced meta-programming techniques, such as staging or macros, that depend heavily on the underlying language technology. In this work we present a language-agnostic solution. We optimize the performance of parsing combinators with specializations of parsing strategies. For each combinator, we analyze the language parsed by the combinator and choose the most efficient parsing strategy. By adapting a parsing strategy for different parser combinators we achieve performance comparable to that of handwritten or optimized parsers while preserving the advantages of parsers combinators.

Context-sensitive parsing for programming languages

“…Thus the extended sequence of moves matches reduction process (24) if 𝑖 > 1 or (23) if 𝑖 = 1. • For 𝑘 + 1 and 𝑖 = (𝑛 + 1) there is at most one move, say 𝛿(𝑞, $) = ⟨𝑞 ′ , $, L⟩, and based on (17) there is reduction ⟨𝑞 ⊳ 𝑋 𝑛 ⟩]] ↦ ⟨𝑞 ′ ∶ 𝑋 𝑛 ⟩]] in  if and only if that move exists in .…”

“…In stateful parsing the global parse-wide state is maintained and manipulated whenever context-free formalism does not suffice. It can be implemented in the classic way, e.g., in a hand-written parser or using a yacc-like tool [23], or using parser combinators [24,25]. ALL( * ) parsing, i.e., adaptive LL( * ) parsing, built into ANTLR4 [26], which is one of the most popular parser generators, provides support for scannerless parsers and predicated ALL( * ) grammars [27].…”

“…Hence, sequence of moves ( 22) can be extended with a move ¢𝑋 1 𝑋 2 … 𝑋 𝑛 𝑞$ ⊢ ¢𝑋 1 𝑋 2 … 𝑞 ′ 𝑋 𝑛 $ and reduction process (24) can be extended with [[⟨𝑋 1 ⟩⟨𝑋 2 ⟩ ⋯ ⟨𝑋 𝑛−1 ⟩⟨𝑞 ′ ∶ 𝑋 𝑛 ⟩]]. Thus the extended sequence of moves matches reduction process (24) for 𝑖 = 𝑛.…”

“…Hence, sequence of moves(22) can be extended with a move𝑞¢𝑋 1 𝑋 2 … 𝑋 𝑛 $ ⊢ ¢𝑞 ′ 𝑋 1 𝑋 2 … 𝑋 𝑛 $ and reduction process (23) can be extended with [[⟨𝑞 ′ ∶ 𝑋 1 ⟩⟨𝑋 2 ⟩ ⟨𝑋 3 ⟩ ⋯ ⟨𝑋 𝑛 ⟩]].Thus the extended sequence of moves matches reduction process(24) for 𝑖 = 1.• For 𝑘 + 1 and 1 ≤ 𝑖 ≤ 𝑛 the head can be moved either right or left but there can be no move available if the input string𝑎 1 𝑎 2 … 𝑎 𝑛 ∉ 𝐿(). Based on (18) there is 𝛿(𝑞, 𝑋 𝑖 ) = ⟨𝑞 ′ , 𝑋 ′ 𝑖 , R⟩ in  if and only if 𝑅 contains either ⟨𝑞 ∶ 𝑋 𝑖 ⟩⟨𝑋 𝑖+1 ⟩ ↦ ⟨𝑋 ′ 𝑖 ⟩⟨𝑞 ′ ∶ 𝑋 𝑖+1 ⟩ if 𝑖 < 𝑛 or ⟨𝑞 ∶ 𝑋 𝑖 ⟩]] ↦ ⟨𝑞 ⊳ 𝑋 ′ 𝑖 ⟩]] if 𝑖 = 𝑛.Hence, sequence of moves (22) can be extended with a move𝑋 0 𝑋 1 … 𝑋 𝑖−1 𝑞𝑋 𝑖 𝑋 𝑖+1 … 𝑋 𝑛+1 ⊢ 𝑋 0 𝑋 1 … 𝑋 𝑖−1 𝑋 ′ 𝑖 𝑞 ′ 𝑋 𝑖+1 … 𝑋 𝑛+1and a reduction process (24) can be extended with[[⟨𝑋 1 ⟩⟨𝑋 2 ⟩ ⋯ ⟨𝑋 𝑖−1 ⟩⟨𝑋 ′ 𝑖 ⟩⟨𝑞 ′ ∶ 𝑋 𝑖+1 ⟩ ⋯ ⟨𝑋 𝑛 ⟩]] if 𝑖 < 𝑛 or [[⟨𝑋 1 ⟩⟨𝑋 2 ⟩ ⋯ ⟨𝑋 𝑛−1 ⟩⟨𝑞 ′ ⊳ 𝑋 ′ 𝑛 ⟩]] if 𝑖 = 𝑛.Thus the extended sequence of moves matches reduction process(24) if 𝑖 < 𝑛 or (24) if 𝑖 = 𝑛. Based on (19) there is 𝛿(𝑞, 𝑋 𝑖 ) = ⟨𝑞 ′ , 𝑋 ′ 𝑖 , L⟩ in  if and only if 𝑅 contains either⟨𝑋 𝑖−1 ⟩⟨𝑞 ∶ 𝑋 𝑖 ⟩ ↦ ⟨𝑞 ′ ∶ 𝑋 𝑖+1 ⟩⟨𝑋 ′ 𝑖 ⟩ if 𝑖 > 1 or [[⟨𝑞 ∶ 𝑋 𝑖 ⟩ ↦ [[⟨𝑞 ′ ⊲ 𝑋 ′ 𝑖 ⟩ if 𝑖 = 1.Hence, sequence of moves (22) can be extended with a move𝑋 0 𝑋 1 … 𝑋 𝑖−1 𝑞𝑋 𝑖 𝑋 𝑖+1 … 𝑋 𝑛+1 ⊢ 𝑋 0 𝑋 1 … 𝑋 𝑖−2 𝑞 ′ 𝑋 𝑖−1 𝑋 ′ 𝑖 … 𝑋 𝑛+1…”

An Innovative Approach to Syntax-Free Interpretation in Functional Programming Languages

“…To address these complexities or to enhance the efficiency of Syntactical Analysis, various strategies have been explored. These include the implementation of combinators [20,21] and the optimization of existing combinators [22][23][24]. In the context of functional programming, the design of such combinators, conceptualized as functions that operate on other functions, introduces additional layers of complexity.…”