Parsing with derivatives

Abstract

We show how to implement a parser combinator library in Haskell capable of parsing arbitrary context-free grammars. The library is based on the work of Might et al. on parsing with derivatives, [15] of which we derive the required results from a single Galois connection. The library improves upon the previous Haskell implementation [5] with regards to completeness, safety and documentation; we can handle any grammar and use generalized tries for memoization instead of unsafe IO. However, our implementation shows severe performance issues. We describe these issues and possible solutions where applicable.

Simplified Parsing Expression Derivatives

“…In typical formulations [3,10,12], the derivative d c (ϕ) is a function from an expression ϕ ∈ X and a character c ∈ Σ to a derivative expression ϕ ′ ∈ X . Formally, L (d c (ϕ)) = {s ∈ Σ * : c s ∈ L(ϕ)}.…”

“…As an additional contribution to the theory of parsing expression grammars, this work includes a formal proof of correctness for its algorithm, a result left as conjecture by authors of previous parsing expression derivative algorithms [12,6]. This paper also presents an extension of the concept of nullability from existing work on derivative parsing [3,10] to PEGs, proving some useful properties of the given presentation while respecting Ford's [5] undecidability results.…”

Derivatives of Parsing Expression Grammars

“…Might et al [13] extended the Brzozowski regular expression derivative [4] to context-free grammars; Adams et al [1] further optimized this approach and proved a cubic worst-case time bound. The derivative parsing algorithm presented here is based on their work, with extensions to account for the lookahead and ordered choice features of PEGs.…”

“…This paper presents a new algorithm for recognizing parsing expression grammars based on the derivative parsing algorithm for context-free grammars introduced by Might et al [13]. Essentially, the derivative of a parsing expression ϕ with respect to a character c is a new parsing expression d c (ϕ) that recognizes everything after the initial c in L (ϕ).…”

“…PEGs are formally defined and these algorithms outlined in Section 3. This paper introduces a PEG recognition algorithm in Section 4 based on the context-free grammar derivative of Might et al [13], which is in turn based on Brzozowski's regular expression derivative [4]. The essential idea of the derivative parsing algorithm is to keep an abstract syntax tree representing the "rest" of the original grammar, iteratively transforming it into a new grammar for every character in the input string.…”