Type-directed, whitespace-delimited parsing for embedded DSLs

Abstract

Domain-specific languages improve ease-of-use, expressiveness and verifiability, but defining and using different DSLs within a single application remains difficult. We introduce an approach for embedded DSLs where 1) whitespace delimits DSL-governed blocks, and 2) the parsing and type checking phases occur in tandem so that the expected type of the block determines which domain-specific parser governs that block. We argue that this approach occupies a sweet spot, providing high expressiveness and ease-of-use while maintaining safe composability. We introduce the design, provide examples and describe an ongoing implementation of this strategy in the Wyvern programming language. We also discuss how a more conventional keyword-directed strategy for parsing of DSLs can arise as a special case of this type-directed strategy.

Safely Composable Type-Specific Languages

“…One can refer to our version of the language as TSL Wyvern when the variant being discussed is not clear. Our work substantially extends and makes concrete a mechanism we sketched in a short workshop paper [23].…”

Reliably composable language extensions

“…Wyvern [35] does support reliable composition of independently developed (syntactic) language extensions, without abandoning parsing in favor of projectional editing.…”

“…Wyvern [35] is the only extensible language system besides Silver, to our knowledge, that supports reliable composition of independently developed language extensions, at least syntactically, without abandoning parsing in favor of projectional editing. However, their approach does not accommodate introducing new analysis of the host language, and so is similar to syntactic macros in being non-interfering, but more limited.…”

“…Wyvern [35] is the only extensible language system besides AbleC, to our knowledge, that supports reliable composition of independently developed language extensions, at least syntactically, without abandoning parsing in favor of projectional editing. It uses a type-directed, white-space sensitive parsing technique that uses indentation to isolate program fragments to be parsed by a language extension.…”

Graceful Language Extensions and Interfaces

“…Wyvern [131] supports nested domain-specific languages within Wyvern code [139,140]. A DSL block is identified by indentation or predefined quoting, and parsing is type-directed: which language a DSL block is written in is determined by the type to which its result is assigned.…”